fgets() vs std::getline() performance
in [C++]
|
From: Pete Becker on 15 Sep 2006 10:13 crhras wrote: > Wow ! I just used two different file IO methods and the performance > difference was huge. Is there something that I am doing wrong? or is > fgets() just that much faster than getline()? > It's easy to implement getline badly. Making it faster requires some thought, but there's no good reason for getline to be significantly slower than fgets. It should be within 10-20 percent. -- -- Pete Author of "The Standard C++ Library Extensions: a Tutorial and Reference." For more information about this book, see www.petebecker.com/tr1book. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: alex on 15 Sep 2006 14:05 crhras wrote: > Wow ! I just used two different file IO methods and the performance > difference was huge. Is there something that I am doing wrong? or is > fgets() just that much faster than getline()? > > Here's the code I used : > > // SLOOOOOOW > // ----------------- > std::string line; > std::ifstream in(filename.c_str()); > > while (std::getline(in, line,'\n')) > { > } > > // FAAAAAASSSSSTTTTT > // --------------------------- > FILE * fp; > fp = fopen(filename.c_str(), "r"); > > while (fgets(line, 512, fp) != NULL) > { > } The code blocks above are not completely equivalent. You can re-test with this block instead: std::ifstream in(filename.c_str()); char line[512]; while (in.getline(line, 512)) { } BTW, specifying precise test conditions may somewhat clarify you point. Alex [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: crhras on 16 Sep 2006 07:49 { Quoted clc++m banner removed. -mod } "Earl Purple" <earlpurple(a)gmail.com> wrote in message news:1158307576.886768.308030(a)d34g2000cwd.googlegroups.com... > > crhras wrote: > >> Wow ! I just used two different file IO methods and the performance >> difference was huge. Is there something that I am doing wrong? or is >> fgets() just that much faster than getline()? >> >> Here's the code I used : >> >> // SLOOOOOOW >> // ----------------- >> std::string line; >> std::ifstream in(filename.c_str()); >> >> while (std::getline(in, line,'\n')) >> { >> } >> >> // FAAAAAASSSSSTTTTT >> // --------------------------- >> FILE * fp; >> fp = fopen(filename.c_str(), "r"); >> >> while (fgets(line, 512, fp) != NULL) >> { >> } > > If you performed the fstream test first for a big file then immediately > did the FILE * test on the same machine with the same file, it is > likely that the O/S had cached something because it was a recently read > file. That might explain the difference in performance. This is what I first thought but I ran the test twice to make sure that caching wasn't a consideration. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: crhras on 16 Sep 2006 07:48 { Quoted clc++m banner removed. -mod } "Thomas Tutone" <Thomas8675309(a)yahoo.com> wrote in message news:1158296928.437104.95290(a)m73g2000cwd.googlegroups.com... > crhras wrote: > >> Wow ! I just used two different file IO methods and the performance >> difference was huge. Is there something that I am doing wrong? or is >> fgets() just that much faster than getline()? >> >> Here's the code I used : >> >> // SLOOOOOOW >> // ----------------- > > > insert the following line here, then rerun your timing test: > > std::ios::sync_with_stdio(false); > >> std::string line; >> std::ifstream in(filename.c_str()); >> >> while (std::getline(in, line,'\n')) >> { >> } > > Please report back on the result after making that change. > > Best regards, > > Tom Thank you for your suggestion. I inserted the line you suggested and it cut around 25% off of the test time but fgets() is still considerably faster than getline(). Here's the test program which I used : ------------------------------------------ time_t start, end; double dif; std::string line; std::ifstream in("c:\\Data\\IVData.csv"); std::ios::sync_with_stdio(false); time(&start); while (std::getline(in, line,'\n')) { } time(&end); in.close(); dif = difftime (end,start); cxMemo1->Lines->Add((AnsiString)"Test 1 has taken " + dif + " seconds."); Result : ------------------ Test 1 has taken 159 seconds. (getline) Test 2 has taken 4 seconds. (fgets) [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ]
From: crhras on 16 Sep 2006 07:55
{ Quoted clc++m banner removed. Wrapped lines fixed up. Because I had the time. ;-) -mod } "crhras" <crhras(a)sbcglobal.net> wrote in message news:TkoOg.1329$e66.993(a)newssvr13.news.prodigy.com... > > > Wow ! I just used two different file IO methods and the performance > difference was huge. Is there something that I am doing wrong? or is > fgets() just that much faster than getline()? > > Here's the code I used : > > // SLOOOOOOW > // ----------------- > std::string line; > std::ifstream in(filename.c_str()); > > while (std::getline(in, line,'\n')) > { > } > > // FAAAAAASSSSSTTTTT > // --------------------------- > FILE * fp; > fp = fopen(filename.c_str(), "r"); > > while (fgets(line, 512, fp) != NULL) > { > } Thank you for the responses. I went back to the drawing board using your suggestions. Some of the results are in this post and others will be posted under the specific newsgroup response which I was testing. First off, the data file used contains 3.5 million text records of varying lengths terminated by '\n'. I reran the tests with some timers to tell exactly how long each case is taking. I ran each test twice in sequence to make sure that caching was not responsible for the time difference. ----------------------------------- // Test 1 while (std::getline(in, line,'\n')) { } ----------------------------------- // Test 2 while (fgets(cline, 512, fp) != NULL) { } Results : Test 1 has taken 203 seconds. Test 2 has taken 5 seconds. Test 1 has taken 201 seconds. Test 2 has taken 4 seconds. [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] |