Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

Problem with fread

Hello,


Since i install Lion i have a problem with fread.

I used xcode 3.2.6 and seeing this problem i desinstall xcode 3.2.6 and install the lastest xcode (4.2).

The problem remain.


Here is the code:


unsigned char *ptr;

size_t len;

FILE *file;


if (!(ptr = calloc(len, sizeof(char))))

{

// error

}

if (fread(ptr, sizeof(char), len, file) != len)

{

// error

}


the problem is :


After ptr is allocated (ok)

if len is > 2147483647 (int size) there is no error, the file is not read. (incredible but true)

else (len <= 2147483647) its ok (file is read).


len is always ok, even for 3Gb or more.


Any ideas ?


Thank you

Mac Pro, Mac OS X (10.7.2), xcode 4.2

Posted on Oct 19, 2011 12:54 PM

Reply
27 replies

Oct 19, 2011 5:14 PM in response to etresoft

> Have you included <stdio.h>?

no (don't need, i never include stdio)


> Are you getting any warnings?

No.

Everything is ok.


It works for years.(same line same code).


it was compiled on xcode 3.2.6 sdk 10.6 with gcc 4.2 and now with llvm 3.0 sdk 10.7.


its since i upgraded to Lion i think , i don't see any change i made in the environnement.


i really don't understand.

Oct 19, 2011 6:08 PM in response to stéphane69

stéphane69 wrote:


> Have you included <stdio.h>?

no (don't need, i never include stdio)

Let that be your first step.


> Are you getting any warnings?

No.


I sincerely doubt that. If something compiles with a warning, it still compiles. Subsequent compilations will skipped, and thus report no warnings or errors, unless you change the source file.


What is happening is that since you haven't included <stdio.h>, the compiler doesn't know the return type is size_t. Instead, it picks the default return type of "int", which screws up the implicit cast to size_t.


Just #include <stdio.h> and all should be fine.

Oct 20, 2011 1:28 AM in response to etresoft

I include stdio.h and Its the same.

i never include stdlib, stdio and others in objective C program , framework include it (apptkit, cocoa).


So i add a test , if len > 2147483647 i loop with smaller len.

This is my first post on apple dev, this program has always worked fine.

and i think its since i recompile it for other modifications in the project under Lion.

Oct 27, 2011 10:06 AM in response to etresoft

Here is a test for the case :


#include <stdio.h>

#include <stdlib.h>

#include <string.h>


int main(int argc, char *argv[])

{

FILE *fin = NULL, *fout = NULL;

char *ptr = NULL;

size_t len;

fpos_t flen;

if(!(fin = fopen(argv[1], "rb")))

{

printf("The input file: %s could not be opened\n", argv[1]);

return -1;

}

if((fout = fopen(argv[2], "rb")))

{

printf("The output file %s already exist\n", argv[2]);

fclose(fin);

return -1;

}

if (!(fout = fopen(argv[2],"wb")))

{

printf("Cannot write output file %s\n", argv[2]);

fclose(fin);

return -1;

}


fseek(fin, 0, SEEK_END);

fgetpos(fin, &flen);

len = flen;

printf("Input file length : %zd\n", len);

fseek(fin, 0, SEEK_SET);


if (!(ptr = malloc(len)))

{

printf("Canot allocate %zd bytes\n", len);

fclose(fin);

fclose(fout);

return -1;

}


if (fread(ptr, sizeof(char), len, fin) != len)

{

printf("Cannot read file\n");

fclose(fin);

fclose(fout);

free(ptr);

return -1;

}


fclose(fin);


if (fwrite(ptr, sizeof(char),len, fout) != len)

{

printf("Cannot write file\n");

fclose(fout);

free(ptr);

return -1;

}

free(ptr);

fclose(fout);


return 1;

}


cc program

and

./pgm fileinput fileoutput


Process the input and outputfile with openssl sha1 for length > 2147483647 and its different.


in fact there is no fread when length > 2147483647

Why ?

Where is the problem in this simple program ?


Thank you

Oct 27, 2011 5:31 PM in response to stéphane69

stéphane69 wrote:


i don't know if it is this program on Lion but if you look here http://www.opensource.apple.com/source/Libc/Libc-186/stdio.subproj/fread.c


r is defined as int

That code has changed little since Chris Torek wrote it for BSD 21 years ago.


"int" is defined to have the same width as a native word on the CPU. Thus, when compiling for x86_64, int is 64-bit.


it worked on snow leopard.

and since lion there is this problem.

not on fwrite, only for fread.


2 GB limit for fread


Perhaps you are compiling it incorrectly or something. Your code works fine on Lion here.


Pele:tmp jdaniel$ clang test.c


Pele:winxp.hdd jdaniel$ /tmp/a.out winxp.hdd.0.{aabbccdd-0204-9357-5189-efb0c034f5ea}.hds out

Input file length : 5943684608


Pele:winxp.hdd jdaniel$ ll

total 46943752

drwxr-xr-x@ 11 jdaniel staff 374B 27 Oct 19:58 .

drwxr-xr-x@ 13 jdaniel staff 442B 24 Oct 15:43 ..

-rw-r--r-- 1 jdaniel staff 2.5K 24 Oct 15:43 DiskDescriptor.xml

-rw-r--r-- 1 jdaniel staff 2.5K 24 Oct 14:34 DiskDescriptor.xml.Backup

-rw-r--r-- 1 jdaniel staff 5.5G 27 Oct 20:10 out

-rw-r--r-- 1 jdaniel staff 0B 15 Dec 2008 winxp.hdd

-rw-r--r-- 1 jdaniel staff 1.2G 1 Nov 2007 winxp.hdd.0.{57f99995-917c-482a-a331-39dd885a8ad9}.hds

-rw-r--r-- 1 jdaniel staff 8.8G 24 Oct 15:43 winxp.hdd.0.{5fbaabe3-6958-40ff-92a7-860e329aab41}.hds

-rw-r--r-- 1 jdaniel staff 1.3G 1 Nov 2007 winxp.hdd.0.{824ff22a-3b01-4bed-ab34-c9b510e34da3}.hds

-rw-r--r-- 1 jdaniel staff 5.5G 7 Oct 2010 winxp.hdd.0.{aabbccdd-0204-9357-5189-efb0c034f5ea}.hds

-rw-r--r-- 1 jdaniel staff 9.4K 24 Oct 14:34 winxp.hdd.drh

Pele:winxp.hdd jdaniel$

Oct 28, 2011 1:08 AM in response to etresoft

> That code has changed little since Chris Torek wrote it for BSD 21 years ago.


Why it is again on the internet on apple source site ?


Where is the fread source code implemented in Lion ?


> Thus, when compiling for x86_64, int is 64-bit.


i can assure you Its difficult with you, ok ?


Int size on 64 bit is :

printf("Size of int types is %d bytes\n",

(int)sizeof(int));

printf("Signed int min: %d max: %d\n",

INT_MIN, INT_MAX);

Size of int types is 4 bytes

Signed int min: -2147483648 max: 2147483647


> Perhaps you are compiling it incorrectly or something. Your code works fine on Lion here.


Have you compared the 2 fingerprints by openssl sha ?


cat the fist line of the files ?


Are they the same files ?


Houston, there is a problem.


So i try to put that on apple bug report and i have a message from apple indicating another bug i have to report.

So i'll try to put that problem on their site.

Oct 28, 2011 1:40 AM in response to etresoft

The program works, there is no error and the 2 files has the same length

Yes i have the same thing.


You have the same length because the pointer is well allocated.


Enter the following command according to your test :


openssl sha winxp.hdd.0.{aabbccdd-0204-9357-5189-efb0c034f5ea}.hds

openssl sha out


Do you have the same fingerprint ?


You can compare the last bytes of the two files by cat command


Do you have the same thing ?


Thank you

Oct 28, 2011 8:30 AM in response to stéphane69

stéphane69 wrote:


> That code has changed little since Chris Torek wrote it for BSD 21 years ago.


Why it is again on the internet on apple source site ?


Where is the fread source code implemented in Lion ?

Actually, your link was to ancient version. This is the source for fread in Lion.


Have you compared the 2 fingerprints by openssl sha ?


No, and I'm not about to. It is irrelevant. It is a buffered read operation. You would never, ever want to read that much data at once and if you did, why would you want to buffer it? It is fairly obvious that Lion supports files larger than 2 GB. It is also obvious that no one would ever use fread (or read) with a buffer that large.


It doesn't matter if it worked in Snow Leopard and doesn't work in Lion. You never should have done it in Snow Leopard to begin with.

Oct 28, 2011 10:24 AM in response to etresoft

> This is the source for fread in Lion.


int r


> No, and I'm not about to. It is irrelevant.


Ok, continue to live in full denial.

Blocking the process.

Its your decision.


> It is a buffered read operation.


This program is a copy, and does not work when file are greater then 2G

Its a fact, and an apple bug.

And it worked with snow leopard and NOT with Lion.


> You would never, ever want to read that much data at once


Its is not persona,l its about a bug in apple product named Lion.

There is a bug and you refuse to recognise that.


> It is fairly obvious that Lion supports files larger than 2 GB.


No and i proved that in this program.

And you refuse to recognize that.

You have the tool to prove it by yourself and you refuse to do it.

Its call live in denial or refusal of reality.


So, delete your original file and rename the output you processed by the program.

As you say : "Your code works fine on Lion here"

DO IT.

It is not because the length is the same than the output file is the same.

The two files you processed are not the same, and you refuse to see that, exposing your incompetence.

Thats a fact.


Had with the lack of anybody qualified in this forum, with minimum skill on developement, i filled a bug report here http://bugreport.apple.com and its difficult because i get the following error :

An error has occurred. Please report the error to Apple Inc. by emailing the error detail to devbugs@apple.com.


and after sending an email to the adress they said to report on http://bugreport.apple.com

creating they own loop.

Fun


i'll try later when they fix that also.


But as i said earlier i modified the program who was working for years on leopard an snow leopard.

Maybe month later on years later they will fix this bug.


Good luck for you in your denial state.


And excuse me having reporting a bug on apple forum and take your time.

I am very sorry,


It was my first post on apple forum and with this awful experience i think the lastest.


Next time i'll fix by myself without reporting here.


> You never should have done it in Snow Leopard to begin with.


Please.

The length is size_t and not int or short int.


Learn programming first.


Bye

Oct 28, 2011 11:26 AM in response to stéphane69

The reason you use buffered I/O is to speed up read and write operations. Each low-level disk access is going to have a fixed amount of overhead. It doesn't matter if you read 12 bytes of 12 million. You use buffered I/O so that when you read 12 bytes, the operating system will read 4096 instead. Then, the next time you need 12 bytes, the data is already in cache waiting for you.


There is no reason to ever issue a single I/O request of 2 GB. There is no reason to issue a single I/O request of more than 64k. If you need to read a variable amount of data, you do so in a loop where you read up to a fixed, smaller amount.


Here is the Snow Leopard version of fread. As you can see, it uses an int as well. People deal with files larger than 2 GB all the time. Nobody ever writes code to read that much data in one command.

Problem with fread

Welcome to Apple Support Community
A forum where Apple customers help each other with their products. Get started with your Apple ID.