How hard is socket programming?
in [MFC]
|
Prev: Runtime Error does not specify program???
Next: windows 7 - imitating touch by sending wm_gesture ?
From: Peter Olcott on 13 Mar 2010 18:58 I want to make a web service, and I don't need any complex protocol such as SOAP. I only need to take a stream of bytes representing a PNG image file as input, and provide UTF-8 text as output. I might have a bunch of small nearly simultaneous requests. These can be processed in their order of arrival. A few of these requests may be multi-megabytes. Is socket programming a good way to go on this?
From: Geoff on 13 Mar 2010 19:35 On Sat, 13 Mar 2010 17:58:02 -0600, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: >I want to make a web service, and I don't need any complex >protocol such as SOAP. I only need to take a stream of bytes >representing a PNG image file as input, and provide UTF-8 >text as output. > >I might have a bunch of small nearly simultaneous requests. >These can be processed in their order of arrival. A few of >these requests may be multi-megabytes. Is socket programming >a good way to go on this? > This sounds like more questions about your OCR appliance. If the clients accessing your service are using the Internet protocol to access your server you do not need to ask this question. You have no choice. If you were coding on the Linux platform you would not need to ask this question, it would be automatically assumed that you are going to use a sockets interface for your service. It would also take about 30 minutes to code and debug a sockets interface and fork logic to pass this kind of data in and out of your OCR process. Outside of your program, one could program a Perl cgi script behind an Apache based web server to copy the received data streams to files with associated cookies or "handles" to the socket sessions hosted by the web server but you would have to process the files and respond to them in such a manner that you could guarantee a reply before the web session expired. The script would handle the socket sessions while your process merely dealt with file I/O (pipes?) to the scripts. Even doing this in Windows with or without MFC, just writing a simple server, one could write a socket server that passed files to/from your OCR program in a few hours. I hope you also realize that your OCR appliance could probably very easily be perverted to defeat web based captcha codes on a vast scale.
From: Joseph M. Newcomer on 13 Mar 2010 22:46 Typically, you would want to look at the HTTP protocol, specifically, SEND and POST requirements, as well as using an encoding such as uuencode to encode your binary data. The typical question is how you want to integrate this with a Web server such as IIS or Aparche. THis would be port 80 (or 8080) protocolos. Otherwise, you are free to obtain a registered port number from IANA (www.iana.org, takes about six weeks, no charge, fill in a simple form), and you are free to send any message in any format to that port #. It does require that you have a static IP address so your program can open a connection to that IP address (obtain via the DNS server mechanisms, e.g., if I had a static IP for flounder.com you could request a DNS resolution of flounder.com and find the one-and-only IP address that represents it; note also that a static IP address usually costs extra, and I chose no to pay the price, so the IP address of flounder.com changes every 24 hours). joe On Sat, 13 Mar 2010 17:58:02 -0600, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote: >I want to make a web service, and I don't need any complex >protocol such as SOAP. I only need to take a stream of bytes >representing a PNG image file as input, and provide UTF-8 >text as output. > >I might have a bunch of small nearly simultaneous requests. >These can be processed in their order of arrival. A few of >these requests may be multi-megabytes. Is socket programming >a good way to go on this? > Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 13 Mar 2010 23:00 See below... On Sat, 13 Mar 2010 16:35:46 -0800, Geoff <geoff(a)invalid.invalid> wrote: >On Sat, 13 Mar 2010 17:58:02 -0600, "Peter Olcott" ><NoSpam(a)OCR4Screen.com> wrote: > >>I want to make a web service, and I don't need any complex >>protocol such as SOAP. I only need to take a stream of bytes >>representing a PNG image file as input, and provide UTF-8 >>text as output. >> >>I might have a bunch of small nearly simultaneous requests. >>These can be processed in their order of arrival. A few of >>these requests may be multi-megabytes. Is socket programming >>a good way to go on this? >> > >This sounds like more questions about your OCR appliance. > >If the clients accessing your service are using the Internet protocol >to access your server you do not need to ask this question. You have >no choice. > >If you were coding on the Linux platform you would not need to ask >this question, it would be automatically assumed that you are going to >use a sockets interface for your service. It would also take about 30 >minutes to code and debug a sockets interface and fork logic to pass >this kind of data in and out of your OCR process. **** Actually, on the server side it is faster than this, because it is just a cgi gateway script; the data is sent up uuencoded or some other suitable encoding, and passed as data to the CGI-invoked program. Of course, you have to code up the CGI code, and that can take a while, but that's going to be fixed overhead no matter what means is used to encode the client data. But an HTTP POST with suitable encoding on the client side is all that is required, plus waiting for the HTTP response, both of which are (a) easy and (b) the same on Windows and linux. **** > >Outside of your program, one could program a Perl cgi script behind an >Apache based web server to copy the received data streams to files >with associated cookies or "handles" to the socket sessions hosted by >the web server but you would have to process the files and respond to >them in such a manner that you could guarantee a reply before the web >session expired. The script would handle the socket sessions while >your process merely dealt with file I/O (pipes?) to the scripts. **** Remember his original question about 500ms? This is part of the overhead that is "beyond the control of the program" that I kept referring to. In fact, the cgi invocation overhead is one of the performance bottlenecks of most Web servers, and Apache is probably the most efficient platform around for handling this (I haven't followed the latest round of tricks, only to note that it now has improved this time considerably) **** > >Even doing this in Windows with or without MFC, just writing a simple >server, one could write a socket server that passed files to/from your >OCR program in a few hours. > >I hope you also realize that your OCR appliance could probably very >easily be perverted to defeat web based captcha codes on a vast scale. **** I didn't want to mention this...but it seems pretty obvious. But we've always known that captcha codes were going to be short-lived... joe **** Joseph M. Newcomer [MVP] email: newcomer(a)flounder.com Web: http://www.flounder.com MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Hector Santos on 13 Mar 2010 23:14
Peter Olcott wrote: > I want to make a web service, and I don't need any complex > protocol such as SOAP. I only need to take a stream of bytes > representing a PNG image file as input, and provide UTF-8 > text as output. > > I might have a bunch of small nearly simultaneous requests. > These can be processed in their order of arrival. A few of > these requests may be multi-megabytes. Is socket programming > a good way to go on this? Hard? Well, thats a subjective topic. For many, it is very simple. If you want a web service, that implies "socket programming" as the WEB is the HTTP protocol with a default port of 80. So you really don't want to learn SOCKET programming but the higher layer HTTP programming. Here is a quick summary: In the WEB, the user (HTTP client) sends a URL command as one of these HTTP commands: GET POST The GET sends all the data on the same URL using NV (name=value) pairs, for example: http://yourdomain.com?n1=v1&n2=v2 If you type a URL in the browser address bar, click a LINK, display a image, this are all done as a GET command with all the input on the URL line. In a POST, the NVPAIRS plus and other data is sent AFTER the request as a BLOCK of data and a POST is done as a FORM submission where you can have a special INPUT control for files uploads. So in the WEB server, the first line that comes in is either a GET or a POST comamnd. This is followed with additional HTTP information called the HTTP REQUEST BLOCK. It has browser type information, plus other Headers. If the request is a POST, then the HTTP block is followed with a blank line plus the posted data and it can come in two format depending on the FORM encoding type. So for GET, the web server sets the request info in the GET line, and for a POST, it needs to read further after the blank line. The above URL as a GET request, the HTTP block could look like this: GET /?n1=v1&n2=v2 HTTP/1.1 Host: yourdomain.com User-Agent: ... browser type. Accept: text/xml,application/xml, .......... Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Cookie: ... any cookies for the domain ... The main ones are: GET /?n1=v1&n2=v2 HTTP/1.1 Host: yourdomain.com as these are typically require by the HTTP standard. A POST will have the HEADER plus the DATA block: POST / HTTP/1.1 Host: yourdomain.com more_headers: ... blank line... data block What is in the data block depends on how you submit it. For you, for your OCR stuff, as a WEB SERVICE, this implies trying to expose a HTTP POST request so you can send the image data. To automatic this, a HTTP client has to basically to the SAME THING a browser will do when you have a FORM with input field of type='FILE'. So before you work on the automation of a web service, you would have to begin this with a interactive UPLOAD form so you can begin writing the backend WEB Server or WEB service to process the upload. So you can have a form, like so: <FORM method="POST" action="/public/handleupload.XXXX" enctype="multipart/form-data"> <PRE> Upload Image: <INPUT TYPE="FILE" NAME="file" /> Click Upload: <INPUT TYPE="SUBMIT" VALUE="Upload" /> </PRE> <INPUT TYPE="HIDDEN" NAME="action" VALUE="OCR"/> </FORM> The ACTION is very important is that is your "SERVICE" that will handle the POST submission. The .XXXX extension is important, more below on that. The hidden control allows you to send Name=Value pairs to help with your request. It is not part of the URL, but part of the data block that is sent in the enctype=:"multipart/form-data" format. You need this for uploads because it has content headers information, like the size and file name. This is also called a MIME format. A submission could look like this: POST /public/handleupload.XXXX HTTP/1.1 Host: hdev1 User-Agent: .......... Accept: ......... Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive Referer: http://hdev1/public/pwe/peter1.wct Content-Type: multipart/form-data; boundary=-----------30631122814371 Content-Length: 3500 -------------30631122814371 Content-Disposition: form-data; name="file"; filename="images.jpg" Content-Type: image/jpeg .... binary data of the image uploaded.... -------------30631122814371 Content-Disposition: form-data; name="action" OCR -----------------------------30631122814371-- So as you can see, the POST has the service URL for the WEB Server to process. The HTTP request block has the type important lines: Content-Type: multipart/form-data; boundary=-----------30631122814371 Content-Length: 3500 that basically says posted data is to follow. Your web server nor service now has to read this using the boundary lines to separate the parts which include the content-types. Now, the .XXXX action is important because it will define and answer your question of how hard or simple it can be. ..XXXX basically tells you what framework or tools you will be using For example: handleupload.ASP <<-- Microsoft ASP backend scripting process handleupload.WCT <<-- Wildcat! (ours) backend scripting process handleupload.JSP <<-- Sun Java backend scripting process handleupload.PHP <<-- A web server with PHP script mapping handleupload.PL <<-- A web server with PERL script mapping handleupload.EXE <<-- A web server with CGI script mapping Overall, the extension tells the web server what to do and who to pass the data too. The good news, is that with these frameworks, except for CGI, you don't have actually handle the HTTP block and data block yourself, the web server already does this and provides a way to "expose" the uploaded information to the script you will be writing. All of these languages will have a way to read the HTTP Request and data information some way. Thats the easy part. The hard part is learning the language syntax. :) Whats good about language like PHP, PL, and EXE CGI, is that most web services offer SCRIPT MAPPING support that will basically have a configuration: extension: .PHP engine: d:\php525\php-cgi.exe extension: .PL engine: d:\perl\bin\perl.exe The EXE extension is special case where the web service will spawn it directly in some secure fashion. But if you have a special web service with its own embedded languages, like .ASP, like our Wildcat system for .WCT, etc, then it its all integrated and very easy to do. So the first thing for you is to basically get yourself a web server first, like IIS if you want to use .ASP. But you can also use IIS to run script maps like PHP, PERL or a EXE cgi. Finally, once you got the web service handling your upload, you now process it and spit out a HTTP response. It can be any any format you want, HTML, PLAIN TEXT, XML, etc. HTTP/1.1 200 OK Date: Sun, 14 Mar 2010 03:47:29 GMT Content-Length: 498 Server: Wildcat/v6.3.453.3 Cache-Control: no-cache Pragma: no-cache Content-Type: text/plain Your OCR UTF8 string is: xxxxxxxxxxxxxxxxxxxxxxxxxx The response format will generally be what the client will handle or what it requested in the submission. Good luck. -- HLS |