From: Joseph M. Newcomer on 12 Apr 2010 12:44
On Sun, 11 Apr 2010 20:00:43 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote:
>"Hector Santos" <sant9442(a)nospam.gmail.com> wrote in message
>> Peter Olcott wrote:
>>> "Hector Santos" <sant9442(a)nospam.gmail.com> wrote in
>>> message news:%23%23iEA1c2KHA.4332(a)TK2MSFTNGP02.phx.gbl...
>>>> Peter Olcott wrote:
>>>>>> So how do your HTTP request get delegated? Four
>>>>>> separate IP address, sub domains?
>>>>>> What happens when they cross domain attempts occur?
>>>>> I would not be using the complex design that you are
>>>>> referring to. One domain, one web server, four OCR
>>>> So you back to a Many to One Fifo queue. And what
>>>> happens with the HTTP responses?
>>> They have another FIFO queue in the opposite direction.
>> So its an synchronized serialization? Or two threads? Do
>> you need any reader/writer?
>The FIFO in the other direction will have four OCR processes
>that are writers and one web server that is the reader.
>>>> By the web server needs to do a +1 and one of the OCR
>>>> has to do a -1.
>>> No not on this memory location. This memory location is a
>>> copy of another memory location that does these things to
>>> the original value. I don't want to slow down the read of
>>> this value by using a lock because it will be read in a
>>> very tight loop.
>> This is so a classic SYNC 101 problem - See Bains, maybe
>> 1st or 2nd chapter.
>Updating the original value is a little tricky, I have books
>on that. Updating the copy only requires that the copy be
>made immediately following an update of the original.
If you are adding or subtracting 1, it is not "a little tricky"; the phrase we like to use
is "trivial", and it describes InterlockedIncrement and InterlockedDecrement. There is
nothing "tricky" about using these.
Otherwise, you would use a CRITICAL_SECTION, which is a spin lock at that level, and is
not tricky at all. It is only slightly less than trivial. And unless you understand what
a CRITICAL_SECTION is, please do not come back and tell us that its overhead is
unacceptable. I don't really feel like explaining the obvious yet once more.
>>> In other words the original memory location may be read
>>> or written to as often as once every 10 ms. It has to be
>>> locked to make sure that it is updated correctly.
>> But you said there is no lock in the paragraph above.
>>> This copy of the memory location could easily be read a
>>> million times a second or more, don't want to slow this
>>> down with a lock.
>> Oy vey!
>>>> No conflicts, no reader/locker locks? No Interlock
>>>> increments and decrements?
>>> As long as a simultaneous read and write can not garble
>>> each other there is no need for any of these things, on
>>> this copy of the original memory location.
>> But do you have both a write and read potention that can
>> occur at the same time or priority inversion due to task
>No priority inversion is possible with this design because
>every low priority job checks at least 100 times a second
>whether it needs to put itself to sleep.
This entire sentence sounds stupid. If you actually implemented it this way, this would
be an incredibly stupid design requiring polling; the correct way is to use a queue with a
semaphore, so there is no need to poll, ever. This sounds like someone who has never
heard of synchronization making pretensions to doing design.
>There is no memory lock on the copy of the original
>NumberOfHighPriorityJobsPending, since this is the only
>thing that is shared across processes, there can be no
>possible lock contention, thus no priority inversion.
How? Exactly HOW do you share a variable across processes (I know how to do it in
Windows, but do you? And I certainly don't know how to do it in linux, although I could
probably discover it if I wanted to waste the time doing the research). But you have
obviously done the research, or you would not assert this is possible, so please explain
how you are mapping address spaces across process boundaries. Are you using shared data
segments established at link time, or memory-mapped files? Can you write the Microsoft C
code to do this in Windows? And if you could, why would you want to, when you should be
using a named semaphore to handle this efficiently (I though performance mattered, and any
design that requires polling a hundred times a second to see if there is something to do
is really amateurish! This kind of design is what gets students in Operating Systems 101
an "F" in their exams. Oh, did I mention I've taught these courses?)
And, of course, even if you had shared data segments, this would be a really stupid
design. A global semaphore would make sense. But I'm not a linux programmer, so I don't
know how linux semaphores work. But I know they exist.
You really need to take this course:
which I teach. It would be a start. It would at least help you dispel most of the myths
you have about how operating systems in general and Windows in particular works.
Each of the three labs in this course in involve concurrency: a multithreaded file
printer, running a child process using anonymous pipes to capture stdout, and a
multiprocess producer-consumer model that uses shared memory queues to pass information
from the produce processes to the consumer processes. Learn about real synchronization
primitives, what processes and threads are like, how a scheduler works, shared data
segments, memory-mapped files, what thread priorities mean, how memory allocation works,
and so on. I WROTE this course, I TEACH this course, I know something about these topics!
And I don't make any more money if you take it because I get paid by the week, not by the
number of students. So this is not a scheme to make more money off anyone. They'll just
slot you into some already-scheduled class.
Joseph M. Newcomer [MVP]
MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 12 Apr 2010 12:55
On Sun, 11 Apr 2010 23:18:07 -0600, Jerry Coffin <jerryvcoffin(a)yahoo.com> wrote:
>In article <LuidnT3tuaC7p1_WnZ2dnUVZ_gCdnZ2d(a)giganews.com>,
>[ ... ]
>> Alternative (a) There are four processes with four queues
>> one for each process. These processes only care about
>> executing the jobs from their own queue. They don't care
>> about the jobs in any other queue. The high priority process
>> is given a relative process priority that equates to 80% of
>> the CPU time of these four processes. The remaining three
>> processes get about 7% each. This might degrade the
>> performance of the high priority jobs more than the next
>There is no such thing with any OS of which I'm aware. At least with
>a typical OS, the highest priority task is the *only* one that will
>run at any given time. Windows (for one example) does attempt to
>prevent starvation of lower priority threads by waking one lower
>priority thread every four seconds.
Sadly, you have to realize that schedulers work like he *imagines*, not like any real
scheduler we have ever seen! So he thinks that schedulers worry about percentages of CPU
time when no scheduler in history has EVER worked this way. But what does reality matter
in these designs?
>Though the specific details differ, Linux works reasonably similarly.
There is an article I cited in this thread on how Linux anti-starvation works, and it is a
really sad design. "Kludge" comes to mind.
>Neither, however, provides any set of priorities that will give
>anything similar to what you've described. It just doesn't exist.
But he won't listen when we tell him these things. This is not how Peter's Fantasy
Operating System would run, so it doesn't matter!
>> Alternative (b) each of the low priority jobs checks to see
>> if a high priority job is in the queue or is notified by a
>> signal that a high priority job is waiting. If a high
>> priority job is waiting then each of these low priority jobs
>> immediately sleeps for a fixed duration. As soon as they
>> wake up these jobs check to see if they should go back to
>> sleep or wake up.
But why not use semaphores? Duh!
>This requires that each of those tasks is aware of its own process
>scheduling AND of the scheduling of other processes of higher
>priority. Worse, without a lot of care, it's subject to race
>conditions -- e.g. if a high priority task shows up, for this scheme
>to work, it has to stay in the queue long enough for every other task
>to check the queue and realize that it needs to sleep, *before* you
>start the high priority task -- otherwise, the task that's supposed
>to have lower priority will never see that it's in the queue, and
>will continue to run
A lack of any understanding of synchronization and queueing theory is essential to this
design succeeding. Unfortunately, real operating systems are not encumbered by this lack
of understanding of how they work, so they will do SOMETHING, just not the something that
PFOS would have done had it existed.
>Bottom line: you're ignoring virtually everything the world has
>learned about process scheduling over the last 50 year or so. You're
>trying to start over from the beginning on a task that happens to be
Actually, we were doing better than this in 1967. But then, we were a bunch of people who
were in touch with reality (read the Multics literature of the era). Nobody was trying to
build fantasy operating systems.
>> These processes could even simply poll a shared memory
>> location that contains the number of high priority jobs
>> currently in the queue. From what the hardware guys have
>> told me memory writes and reads can not possibly garble each
>This has the same problem outlined above. It adds the requirement for
>a shared memory location, and adding polling code to the OCR tasks.
>See above about ignoring what the world has learned about process
>scheduling over the last 5 decades or so.
Why? PFOS can do this, so why shouldn't a real operating system? Never mind that this
design would fail and Operating Systems 101 design course! And we knew how to do this
better in 1968 (read Dijkstra's "Cooperating Sequential Processes" paper
http://userweb.cs.utexas.edu/users/EWD/transcriptions/EWD01xx/EWD123.html) which is the
basis for nearly all modern operating systems. In fact, the notation he uses is
suspiciously like OpenMP or the PPL (Parallel Patterns Library, VS 2010). This was not an
accident that these systems adopted his ideas. That paper, which we saw in preprint form
at CMU, was about 1968, and was my introduction to concurrent programming.
>[ ... ]
>> Neither of these designs has any of the behavior that you
>No -- they're substantially worse. At least all that did was
>occasionally start a lower-priority task out of order.
>[ ... ]
>> I already figured out a way around that. Everyone must have
>> their own user account that must be created by a live human.
>> All users are always authenticated against this user
>> account. I don't see any loopholes in this on single form of
>Not even close, and you clearly don't understand the problem at all
>yet. The problem is that to authenticate the user you've *already*
>created a thread for his connection. The fact that you eventually
>decide not to do the OCR for him doesn't change the fact that you've
>already spawned a thread. If he makes a zillion attempts at
>connecting, even if you eventually reject them all, he's still gotten
>you to create a zillion threads to carry out the attempted
>authentication for each, and then reject it.
The end result being a successful Denial-of-service (D-O-S) attack on his site!
>Of course, that also ignores the fact that doing authentication well
>is non-trivial itself.
Not in PFOS!
Joseph M. Newcomer [MVP]
MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Joseph M. Newcomer on 12 Apr 2010 13:22
On Mon, 12 Apr 2010 10:05:29 -0500, "Peter Olcott" <NoSpam(a)OCR4Screen.com> wrote:
>"Jerry Coffin" <jerryvcoffin(a)yahoo.com> wrote in message
>> In article
>> NoSpam(a)OCR4Screen.com says...
>> [ ... ]
>>> Alternative (a) There are four processes with four queues
>>> one for each process. These processes only care about
>>> executing the jobs from their own queue. They don't care
>>> about the jobs in any other queue. The high priority
>>> is given a relative process priority that equates to 80%
>>> the CPU time of these four processes. The remaining three
>>> processes get about 7% each. This might degrade the
>>> performance of the high priority jobs more than the next
>> There is no such thing with any OS of which I'm aware. At
>> least with
>> a typical OS, the highest priority task is the *only* one
>> that will
>> run at any given time. Windows (for one example) does
>> attempt to
>> prevent starvation of lower priority threads by waking one
>> priority thread every four seconds.
>The alternative that you show quoted above is called time
>slicing and has been available for many decades.
And therefore timeslicing takes ZERO overhead, right? And you are still thinking that
mucking with thread priorities is going to result in a flawless design (and if you start
ranting about how "thread" and "processes" are different, you will only confirm that you
are completely and utterly clueless)
>> Though the specific details differ, Linux works reasonably
>> Neither, however, provides any set of priorities that will
>> anything similar to what you've described. It just doesn't
> Here is is:
>I will have to run my own tests to see how the process
>priority number map to the relative process priorities that
>I provided above. Ultimately the schedule algorithm boils
>down to essentially the frequency and duration of a time
>slice. There is no need to map to the exact percentage
>numbers that I provided.
As I said, clueless. I knew about 'nice' in 1975, and also understood what the
consequences are. You are reading a superficial article that does NOT really tell you
what is happening to the thread.
>>> Alternative (b) each of the low priority jobs checks to
>>> if a high priority job is in the queue or is notified by
>>> signal that a high priority job is waiting. If a high
>>> priority job is waiting then each of these low priority
>>> immediately sleeps for a fixed duration. As soon as they
>>> wake up these jobs check to see if they should go back to
>>> sleep or wake up.
>> This requires that each of those tasks is aware of its own
>> scheduling AND of the scheduling of other processes of
>> priority. Worse, without a lot of care, it's subject to
>> conditions -- e.g. if a high priority task shows up, for
>> this scheme
>> to work, it has to stay in the queue long enough for every
>> other task
>> to check the queue and realize that it needs to sleep,
>> *before* you
>> start the high priority task -- otherwise, the task that's
>> to have lower priority will never see that it's in the
>> queue, and
>> will continue to run.
>My scheduler will signal all of the low priority jobs that
>they need to sleep now. When the high priority queue is
>empty, and all of the high priority jobs are completed the
>low priority jobs get a signal to wake up now.
Oh, a whole NEW mechanism! Has anyone said "this is without a doubt the dumbest design I
have ever seen"? Well, it is. You really don't understand operating systems, because the
ideas you state here show a fundamental confusion about the right way to build
I presume you mean the linux 'signal' operation. Have you really studied the problems
involved in using it?
>> Bottom line: you're ignoring virtually everything the
>> world has
>> learned about process scheduling over the last 50 year or
>> so. You're
>> trying to start over from the beginning on a task that
>> happens to be
>> quite difficult.
>I see no other way to provide absolute priority to the high
>priority jobs (paying customers) over the low priority jobs
>(free users). Also I see no way that this would not work
>well. If I get enough high priority jobs that the lower
>priority jobs never ever get a chance to run that would be
>fantastic. The whole purpose of the free jobs is to get more
The notion that there is an 'absolute' priority is part of your problem. I think you
don't really understand the basic mechanisms of the operating system. I would not waste
time trying to build such a complex solution to a trivial problem; I would build the
obvious, trivial solution and get the performance I needed, without introducing complex
mechanisms that do not need to exist.
>If you see something specifically wrong with this approach
>please point out the specific dysfunctional aspect. I see no
>possible dysfunctional aspects with this design.
The whole notion of having to "signal" the "low priority threads" to "sleep" and to "wake
up" is ill-founded. You have a LOT of problems you are creating for yourself in how you
would implement this. You seem to think that such signaling is trivial; it is not. There
are fundamental race conditions which you have not comprehended, which will potentially be
fatal to such designs (read about the "wakeup-waiting" flag that Multics had to implement
to handle these race conditions...oh, wait a minute, that is a classic paper by Jerry
Salzer, and therefore would not be part of what you read, which is only the first half of
some student text on operating systems)
>> This has the same problem outlined above. It adds the
>> requirement for
>> a shared memory location, and adding polling code to the
>> OCR tasks.
>> See above about ignoring what the world has learned about
>> scheduling over the last 5 decades or so.
>I already changed this to signals, they don't waste as much
Really? Do you know how 'signal' actually works? And what the race conditions are?
>> No -- they're substantially worse. At least all that did
>> occasionally start a lower-priority task out of order.
>I see no possible sequence of events where this would ever
>occur, if you do please point it out detail by detail.
Look for what happens if a signal comes in between two instructions that are involved in
the synchronization. Most of these techniques are subject to serious race-condition
problems, which is why 'signal' is not even supported in Windows--too many problems, and
people who use it create programs that won't work correctly.
>>> I already figured out a way around that. Everyone must
>>> their own user account that must be created by a live
>>> All users are always authenticated against this user
>>> account. I don't see any loopholes in this on single form
>> Not even close, and you clearly don't understand the
>> problem at all
>> yet. The problem is that to authenticate the user you've
>> created a thread for his connection. The fact that you
>> decide not to do the OCR for him doesn't change the fact
>> that you've
>> already spawned a thread. If he makes a zillion attempts
>> connecting, even if you eventually reject them all, he's
>> still gotten
>> you to create a zillion threads to carry out the attempted
>> authentication for each, and then reject it.
>Block IP long before that.
WHat IP? You clearly have some bizarre notion that the IP is unforgeable. If I wanted to
do a D-O-S on your site, I'd change the IP with every transmission. You have some awfully
naive views about the goodness of the world. Or are we talking about Peter's Fantasy
Internet, where everyone is well-behaved, there are not D-O-S attacks, and nobody EVER
emails a virus?
>> Of course, that also ignores the fact that doing
>> authentication well
>> is non-trivial itself.
Joseph M. Newcomer [MVP]
MVP Tips: http://www.flounder.com/mvp_tips.htm
From: Jerry Coffin on 12 Apr 2010 13:25
In article <370a869f-3f56-4521-ad9d-66784df27a12@
5g2000yqj.googlegroups.com>, sant9442(a)gmail.com says...
> > This is a scenario where something like Amazon's EC2 would work well
> > -- you pay only for processor time you actually use, but if you get a
> > big job, it can run your task on dozens or even thousands of
> > processors so (for example) all the pages from a book are OCRed in
> > parallel, the results put back together, and your customer gets his
> > result back quickly. Then your system goes idle again, and you quit
> > paying for *any* processor time until another task comes along.
> Interesting proposal.
> But this won't eliminate coding requirements? He only leverages the
> computing power and scaling needs which is good as one less (big)
> issue to deal with, but he still needs to frame his code around the
> same principles. In fact, this current framework of Many Threads to 1
> FIFO queue might cost him more based on my readings. He will need to
> make it dynamic loading for EC2 to minimize his cost.. Right?
No, it wouldn't eliminate coding, but it would let him concentrate
primarily on the OCR engine proper. I haven't written anything for it
myself, but the general idea wouldn't be a lot different than he's
working on right now -- you build a complete system on Linux, create
a virtual machine image of it, and upload it to Amazon's servers.
They charge you a little for storage and per-processor time for the
The difference from what he's doing right now is that instead of
being restricted to running on one single-core processor, it'll
automatically load up multiple instances of his virtual machine onto
multiple processors and run them all concurrently when/if there's
enough load to justify it. They provide a billing system that already
works, load balancing, more processing time than you could hope to
put to good use, geographically distributed top-tier data centers,
etc. In short, they already take care of a lot of what he's worrying
about, and quite a bit more that he doesn't know to worry about yet.
Of course, Amazon isn't the only possibility -- Google App Engine and
Windows Azure do the same sorts of things. The obvious difference is
that EC2 let's you put your "stuff" in a VM image, so his current
code should remain usable. Google App Engine requires that you code
in Python or Java. IIRC, Windows Azure supports .NET, so he _might_
be able to use existing code, but it's not at all certain.
From: Hector Santos on 12 Apr 2010 13:39
Peter Olcott wrote:
>>>> if (NumberOfHighPriorityJobsPending !=0)
>>> 20 milliseconds
>> Ok, fair enough, you honestly cited the wrong function,
>> but we get
> No I didn't
> I did provide the wrong parameters, it should have been
> nanosleep(0, 20000000)
> 20000000 nanoseconds = 20 ms
Hence, you used the wrong function for the data you provided - period.
>> Who does the HTTP delegation? Or will you have FOUR
>> different web servers with four different FORM pages?
>> How do you control the cross domain security and business
>> thread entry points where someone is trying to cheat you
>> out of a few dollars?
> (1) Everyone authenticates
How do does this take?
> One physical CPU with a single core and two hyperthreads.
Is that possible?
>> Do you see the conflicts all based on your own
>> ever-changing descriptions?
> No but I do see the conflicts based on your misreading what
> I said.
You say 10 different things each message! and you still have a
conflict with each one!
>> A single CORE does not have HYPERTHRREADING!!! That is
>> only possible with a multi-core CPU.
> Not according to Intel terminology. Hyperthreading was what
> Intel came out with before they came out with multiple
> cores. Now that they came out with multiple cores, some of
> these multi-core CPUs also have two hyperthreads per core,
> and some do not.
Show a reference where it says 1 CPU with one CORE offers
Hyperthreading. Maybe the mix up is what CORE means.
You can not have hyperthreading without atleast TWO LOGICAL
processors. These can be called cores.
Hyper-threading is an Intel-proprietary technology used
to improve parallelization of computations (doing multiple
tasks at once) performed on PC microprocessors. For each
processor core that is physically present, the operating
system addresses two virtual processors, and shares the
workload between them when possible. Hyper-threading requires
not only that the operating system support multiple processors,
You are confusing multi-core with two physically separate multi-cpu
and you are confusing the marketing usage and changes with "HT" versus
dual vs multi-core. In other words, when intel use HT - it means the
processor has multiple cores. There is no such thing AFAIK as a 1 CPU
with 1 CORE that offers HyperThreading.
A multi-core processor is a processing system composed of
two or more independent cores.
and from the horse mouth:
Today's Intel� Hyper-Threading Technology (Intel� HT Technology)
delivers thread-level parallelism on each processor resulting in
more efficient use of processor resources....
Note it says and its guilty also of using PROCESSOR in lieu of CORE,
but nonetheless it says EACH PROCESSOR. Not ONE processor!
>> So you will have a Multiple-CORE machine where each
>> process affinity is set, all a SINGLE CPU machine where
>> they is no hyperthreading?
> No that is not what I said and yet another false assumption
> on your part.
See above. Do you know how to read anything even technical documents
on the web?
>>> This is not a given, but, using time to synchronize is
>>> not the best idea.
>> It is never a good idea. So why did you show it?
> It would have probably worked well enough, but, changing it
> is better.
It would of worked perfectly until it broke - Classic SYNC 101 error!
You are no exception.
> I always design completely before I begin coding,
and you do it so poorly you never get anything done. You would be
totally lost without the help of people on the net.
>> Not related to your design solution, but a cause
>> contributing towards your design problem.
> So then are there are specific dysfunctional aspects with
> the simple time slicing that I proposed immediately above
> that you can point out?
First, you are not time slicing, you are yielding the rest of your
quantum when you sleep. The OS time slicing YOU. You need to learn
how to use the terms and ideas correctly.
Do you mean signaling like I TOLD YOU to use (you did not propose it)?
It depends on what you believe is best performance wise. Are you
attempting to throttle a request? Slow it down? Put incoming low
priority request to "sleep" until the 1 single fifo processor is
complete with the current high priority request?
If you want to throttle the new thread (slow it done) and give more
quantum to the other high priority thread, you can yield (give up your
If you want block it to give even MORE quantum to the HP thread, then
you can wait on a kernel object which will be a VERY efficient "SLEEP"
until it is signaled to wake up.
IMO, with my experience, it is better to leave your threads alone and
only boost the high priority thread and see how that performs. If
you see a lower thread hogging up your high priority, then you
probably to make it wait for HP completion or throttle it. But the
whole point of all this is you don't really know until you see this
all in action with a LOADING - how your request are coming in various
rates and frequencies.
There is no one equation here and 99.99% you need to make this
configurable or at some intelligence to make it learn.
The lesson here is not to lock yourself into one way that it puts
constraints on all the other parts and as a result, those other parts
puts constraints on these tunning factors.
I can give you one recent specific experience. POP3 server. We tried
to increase the priority on one thread for an important action it is
terribly affected the rest of the server. By far, by letting the OS do
its job with scheduling, we obtain the best performance across the
board. But trying to give the HP thread full power, you might find
the the rest is not satisfactory, even its a low tier request item.
Remember, first impressions count - if they see a poor slow trial,
they might even bother subscribing to your service.
>> Well, until we get straight what you THINK a "single core"
>> processor means, it depends. I think you mean each process
>> is assigned a CPU affinity on a multi-core machine.
> PENTIUM 4, it might not even have hyperthreading.
Ok, so you going with a pure single processor. Hey, that might be
better, but you don't know until you tried a multi-core or multi-cpu
with multiple threads. Remember what you will be saving, context
>> But as I pointed out, they are not as isolated as you
>> - HTTP request delegator or proxy?
> The web server does this.
>> - common customer database for authentication and ACL
>> (access control list) or four separate databases?
> single customer database
>> - One log for each one or one single log?
> A single log, and additionally a separate log for each paid
> job stored in the customer database with the output data.
So you realize these need syncing. The logging can use fast appending,
but the the customer database will need syncing specially if using
SQLITE3. For this, go MYSQL to avoid the full locks SQLITES provides.
>> So each OCR can handle any job type. Fine. But you need a
>> single source manager as indicated above which negates
>> your design.
> Not at all. The web server delegaes the jobs to queues.
That is what what I said and you will be load balancing othewise YOU
will be hosed.
>> Don't forget the HTTP response requirements - which is
>> another bottle neck since you have to update/log state
> OCR process tells the web server when a specific job is
Well requires the WEB SERVER threads to wait.
>>> As long as the ROW lookup maps to the file byte offset we
>>> are good.
>> Then get another data manager because SQLITE will not give
>> you this.
> And you know this how?
Because we, many of customers and 3rd party developers use SQLITE3 in
many our products. It uses ISAM/BTREE behind the scenes but that
doesn't mean you have access to - you will be violating the integrity.
Now, that said, if you have the cojones, you can use whats called
SQLITE3 VFS 'Virtual File System' and VFM, Virtual File Methods which
allows you to do custom designs into the HOOKS of the SQLITE3 api.
But why would anyone do this for SQL is beyond me. The only
legitimate reason you might want to use this and the SQLITE3 designers
will advise you against it, is to begin changing its locking scheme.
The fact you think you need SQL with file objects access means you
clearly are applying it wrong.
>>> If the ROW lookup must read and maintain an index just to
>>> be able to get to the rows in sequential order, this may
>>> not be acceptable.
>> Huh, you SELECT gives you the order you want - you declare
>> it in the SELECT. But its not a FILE OFFSET thing.
> Eventually it has to always be a file byte offset thing in
> the underlying physical implementation because Unix/Linux
> only knows about files in terms of a sequences of bytes. If
> I tell the file manager that I want record number 12463 and
> each record has exactly 12 bytes then it must seek (12463 -1
> * 12) byte offset.
First, you are giving LINUX/UNIX a bad reputation with this primitive
thinking. Second, SQL is not a packed record concept. Sorry. You
want an ISAM or even varying record concept. But not SQL. Wrong TOOL.
Now, can you create a TABLE with a field that is a BLOB, where you
save a fixed length binary field?
SURE! you can do that. But you will be accessing sql FIELDS, not
direct file byte offsets.
>> Since your HTTP request are not bulk, each one is handled
>> separated. At the rate you expect for the TPS, you will be
> It will merely be reduced to the maximum number of
> transactions that can be written to disk. Most of this time
> would be disk drive seek time. My cheap drive has about 9 ms
> seek time that's 111 seeks per second. So SQLite might be a
> bottleneck unless a single transaction takes a single seek.
Gawd, you such a bad designer.
>> Again, it is explained to you, yet you refused to listen.
>> SQLITE is not what you want as a shared SQL database among
>> multiple WEB and OCR accessors.
> The transaction write time may still be a bottleneck for any
> other SQL processor.
Not with SQL or database systems that allow for SQL row/record/cursor
level I/O access.
>> You don't know what you are talking about. By definition,
>> Record numbers are not out of sequence. And what are the
>> indices you are
> Record numbers can and indeed are capable of being placed
> out-of-sequence in SQLite.
Not in view YOU get them. You can get it ascending or descending.
By definition, ROW NUMBERS are sequential! Now the insider, it can
have fragmentation, but the PURPOSE of SQL if to give you the records
in the order you want. Not in how its STORED!
In other words, if you did:
SELECT * FROM PETER_TABLE
you are not going to see row ids 10, 100, 20, 5, 2, 6, 7. You will see:
Its not like you think where it uses the first EMPTY ISAM RECORD that
was last deleted and before you packed the database. Sorry Charlie.
It is not the NON-SQL Databasing you did for 10 years, 10 years ago.
>> referring to? What (fields) are they based on?
> The record number.
So you make the record number the primary key? you could, by why is
the reason again?
>> You see, "Single Core?" you are using the wrong terms.
>> Yet, joe and I have a good idea of what you trying to say.
> PENTIUM 4
> I have been using the right terms all along and you keep
> insisting on ignoring them.