ABSTRACT

The function returns an unsigned char cast to an int, but the return value is assigned to a char type.

EXPLANATION

When an unsigned character cast to an integer is assign to a signed character, its value might be indistinguishable from EOF.

Example 1: The code below reads a character and compares it to EOF.


char c;

while ( (c = getchar()) != '\n' && c != EOF ) {
...
}


In this case, the return value from getchar() is cast to a char and compared to EOF (an int). Assuming c is a signed 8-bit value and EOF is a 32-bit signed value, then if getchar() returns a character represented by 0xFF, the value of c will be sign extended to 0xFFFFFFFF in the comparison to EOF. Since EOF is typically defined as -1 (0xFFFFFFFF), the loop will terminate erroneously.

REFERENCES

[1] Standards Mapping - Security Technical Implementation Guide Version 3 - (STIG 3) APP3550 CAT I

[2] Standards Mapping - Common Weakness Enumeration - (CWE) CWE ID 398

[3] FIO34-C. Use int to capture the return value of character IO functions CERT