The function returns an unsigned char
cast to an int
, but the return value is assigned to a char
type.
When an unsigned character cast to an integer is assign to a signed character, its value might be indistinguishable from EOF
.
Example 1: The code below reads a character and compares it to EOF
.
char c;
while ( (c = getchar()) != '\n' && c != EOF ) {
...
}
getchar()
is cast to a char
and compared to EOF
(an int
). Assuming c
is a signed 8-bit value and EOF
is a 32-bit signed value, then if getchar()
returns a character represented by 0xFF, the value of c
will be sign extended to 0xFFFFFFFF in the comparison to EOF
. Since EOF
is typically defined as -1 (0xFFFFFFFF), the loop will terminate erroneously.[1] Standards Mapping - Security Technical Implementation Guide Version 3 - (STIG 3) APP3550 CAT I
[2] Standards Mapping - Common Weakness Enumeration - (CWE) CWE ID 398
[3] FIO34-C. Use int to capture the return value of character IO functions CERT