Designing fgetline - a perspective
in [General Programming]
|
Prev: problem analysis chart
Next: Please help!
From: Malcolm McLean on 14 Oct 2007 02:52 "Richard Harter" <cri(a)tiac.net> wrote in message > I disagree. A common usage is read all of the lines of a file > and keep them in memory. If the buffers holding the lines are > all maximum size then you get memory bloat. The purpose of > maxsize is to serve as a sanity check. > That's a good point. If we're going to support reading the whole file, efficiently, which is worth doing, it means we should also support reading in a file structure in only a few lines. while(1) { fg = fgetline(fptr, 0, 0, FG_MAXSIZE); if(fg.status != fg_normal) { if(fg.status == end_of_file) break; /* error handling code */ } temp = realloc(lines, (N+1) * sizeof(char *)); if(!temp) { /* error handling code */ } lines = temp; lines[N] = fg.line; N++; } it's not an utter disaster, but it is too much. How about. while(line = fg_getline(fptr, 0, 0, FG_MAXSIZE)) { temp = realloc(lines, (N+1) * sizeof(char *)); if(!temp) { /* error-handline code here */ } lines = temp; lines[N] = line; N++; } /* error handler if fg_getline encounters a problem */ if(fg_error(fptr)) { switch(fg_error(fptr)) { } } Don't really like that either. We've got the while() actually testing the condition, and we've moved some but not all of the error handling code out of the loop, but it still isn't a boiler-plate idiom. The other alternative is to have two functions, one for getting a single line, one for reading a whole file. -- Free games and programming goodies. http://www.personal.leeds.ac.uk/~bgy1mm
From: CBFalconer on 14 Oct 2007 11:23 Malcolm McLean wrote: > "Richard Harter" <cri(a)tiac.net> wrote in message > >> I disagree. A common usage is read all of the lines of a file >> and keep them in memory. If the buffers holding the lines are >> all maximum size then you get memory bloat. The purpose of >> maxsize is to serve as a sanity check. > > That's a good point. If we're going to support reading the whole > file, efficiently, which is worth doing, it means we should also > support reading in a file structure in only a few lines. > .... snip code example ... Here is another example, from the testing suite for ggets. Note the simplicity and safety of capturing the entire file in memory (assuming sufficient memory exists). No excess memory is used. ggets.zip is available at: <http://cbfalconer.home.att.net/download> /* File freverse.c - testing ggets */ #include <stdio.h> #include <stdlib.h> #include <string.h> #include "ggets.h" typedef struct node { struct node *next; char *ln; } node, *nodeptr; /* ======================= */ /* reverse string in place */ /* return length of string */ size_t revstring(char *stg) { char *last, temp; size_t lgh; lgh = strlen(stg); if (lgh > 1) { last = stg + lgh; /* points to '\0' */ while (--last > stg) { temp = *stg; *stg++ = *last; *last = temp; } } return lgh; } /* revstring */ /* ======================= */ /* reverse file */ int main(void) { char *line; nodeptr root, temp; int lncount, chcount; fprintf(stderr, "Reversing stdin to stdout\n"); root = NULL; lncount = chcount = 0; while (0 == ggets(&line)) { if (NULL == (temp = malloc(sizeof *temp))) { fprintf(stderr, "Out of memory\n"); return EXIT_FAILURE; } temp->next = root; chcount += revstring(line); lncount++; temp->ln = line; root = temp; } /* file captured, and in reverse ordered list */ while (root) { (void)puts(root->ln); temp = root; root = root->next; free(temp->ln); free(temp); } fprintf(stderr, "%d chars in %d lines\n", chcount, lncount); return 0; } /* main */ /* END freverse.c - testing ggets */ -- Chuck F (cbfalconer at maineline dot net) Available for consulting/temporary embedded and systems. <http://cbfalconer.home.att.net> -- Posted via a free Usenet account from http://www.teranews.com
From: �a/b on 19 Oct 2007 02:49 In data Sun, 14 Oct 2007 11:23:16 -0400, CBFalconer scrisse: >/* File freverse.c - testing ggets */ > >#include <stdio.h> >#include <stdlib.h> >#include <string.h> >#include "ggets.h" > >typedef struct node { > struct node *next; > char *ln; >} node, *nodeptr; > >/* ======================= */ >/* reverse string in place */ >/* return length of string */ >size_t revstring(char *stg) >{ > char *last, temp; > size_t lgh; > > lgh = strlen(stg); > if (lgh > 1) { > last = stg + lgh; /* points to '\0' */ > while (--last > stg) { > temp = *stg; *stg++ = *last; *last = temp; > } > } > return lgh; >} /* revstring */ > >/* ======================= */ >/* reverse file */ >int main(void) >{ > char *line; so it is ok even if here line!=0 ? > nodeptr root, temp; > int lncount, chcount; > > fprintf(stderr, "Reversing stdin to stdout\n"); > root = NULL; > lncount = chcount = 0; > while (0 == ggets(&line)) { > if (NULL == (temp = malloc(sizeof *temp))) { > fprintf(stderr, "Out of memory\n"); > return EXIT_FAILURE; > } > temp->next = root; > chcount += revstring(line); > lncount++; > temp->ln = line; > root = temp; > } > > /* file captured, and in reverse ordered list */ > > while (root) { > (void)puts(root->ln); > temp = root; > root = root->next; > free(temp->ln); > free(temp); > } > fprintf(stderr, "%d chars in %d lines\n", chcount, lncount); > return 0; >} /* main */ >/* END freverse.c - testing ggets */ > >-- > Chuck F (cbfalconer at maineline dot net) > Available for consulting/temporary embedded and systems. > <http://cbfalconer.home.att.net>
From: Richard Harter on 19 Oct 2007 12:01 On Sun, 14 Oct 2007 07:52:17 +0100, "Malcolm McLean" <regniztar(a)btinternet.com> wrote: > >"Richard Harter" <cri(a)tiac.net> wrote in message >> I disagree. A common usage is read all of the lines of a file >> and keep them in memory. If the buffers holding the lines are >> all maximum size then you get memory bloat. The purpose of >> maxsize is to serve as a sanity check. >> >That's a good point. >If we're going to support reading the whole file, efficiently, which is >worth doing, it means we should also support reading in a file structure in >only a few lines. > >while(1) >{ > fg = fgetline(fptr, 0, 0, FG_MAXSIZE); > if(fg.status != fg_normal) > { > if(fg.status == end_of_file) > break; > /* error handling code */ > } > temp = realloc(lines, (N+1) * sizeof(char *)); > if(!temp) > { > /* error handling code */ > } > lines = temp; > lines[N] = fg.line; > N++; >} > >it's not an utter disaster, but it is too much. > >How about. > >while(line = fg_getline(fptr, 0, 0, FG_MAXSIZE)) >{ > temp = realloc(lines, (N+1) * sizeof(char *)); > if(!temp) > { > /* error-handline code here */ > } > lines = temp; > lines[N] = line; > N++; >} >/* error handler if fg_getline encounters a problem */ >if(fg_error(fptr)) >{ > switch(fg_error(fptr)) > { > } >} > >Don't really like that either. We've got the while() actually testing the >condition, and we've moved some but not all of the error handling code out >of the loop, but it still isn't a boiler-plate idiom. > >The other alternative is to have two functions, one for getting a single >line, one for reading a whole file. To be honest, I don't much like either alternative. However that may merely be a case of I like my wheel better. I have come up with an alternative that I like, but I haven't gone through the work of writing it all up. The essence is to have three routines, an open, a read, and a close, along with an include file, and an opaque struct to hold state. The usage looks something like this: #include "rdfline.h" .... struct rdf_info *rdf; /* Opaque, allocated by fg_open */ FILE * fptr; char * line; size_t len; int status; ... if (!rdf_open(rdf,fptr,&line,&len,&status,maxsize,flags)) { /* Can't open - probably fopen failed */ /* Do something about problem and goaway */ } ... while (rdf_read(rdf)) { /* Optional error test code */ /* Do stuff with line */ } /* Optional error test code */ rdf_close(rdf); The flags variable uses various bits to set flags that control things like error handling, memory management, and whether to produce clean or transient copies. Passing in a 0 uses the defaults which are: (a) The package does nothing about errors (writing to a log file, etc) except set an error bit in the status word. (b) A clean copy is produced; the user is responsible for freeing it. (c) rdf_read reads as much as it can and doesn't worry about a premature EOF or memory failure - set a flag bit if you want it to care. More later. Richard Harter, cri(a)tiac.net http://home.tiac.net/~cri, http://www.varinoma.com In the fields of Hell where the grass grows high Are the graves of dreams allowed to die
From: Malcolm McLean on 20 Oct 2007 01:35
"CBFalconer" <cbfalconer(a)yahoo.com> wrote in message > Here is another example, from the testing suite for ggets. Note > the simplicity and safety of capturing the entire file in memory > (assuming sufficient memory exists). No excess memory is used. > ggets.zip is available at: > The ggets() interface isn't too bad at all. I didn't think so before this thread, but looking at the alternatives, its deficiencies are quite minor. -- Free games and programming goodies. http://www.personal.leeds.ac.uk/~bgy1mm |