Prev: How to define which user can "sudo" and for which commands ?
Next: single disk to raid 1
From: MatthiasH on 13 Jul 2010 05:09
Dear all,

I am sorry that this topic does not 100% fit this group, but in lack of
a better alternative I'll still give it a try. The problem I am
describing is predominant in Linux/Unix/MacOSX, so perhaps you can help
me out with some expertise.

I am trying to emulate a FAT16 USB device on a microcontroller with
ridiculously little RAM. Read and write accesses are directly forwarded
to external chips such as EEPROMs. The FAT part is hardcoded. So far, I
have expected the location of the files to remain fixed - whenever the
host writes to (or reads from) a specific sector, it would be mapped to
the external memory. However, when it comes to writing to an existing
file, it appears as if many OSes, instead of just overwriting the
existing data, make a new file system entry, write the data to an unused
cluster, and delete the old entry. Of course, then the data will be
located at a different sector address, and my dumb little device will be
confused. What I now want to try is to keep track of the OS' changes in
the FAT structure and expect the data to be written at the modified
location. However, there are some cases where this is plain impossible,
and I really need to know them in order to come up with an
as-general-as-possible solution.

So here's my question to all you Linux experts out there: Making a new
file should take three steps: allocate clusters in the FAT, enter file
in the directory structure, write data. Can anyone help me with some
information from the Linux kernel that would help me understand in which
order this is done, e.g. when calling fopen() from C/C++?

Thanks,
Matthias
From: The Natural Philosopher on 13 Jul 2010 05:30
MatthiasH wrote:
> Dear all,
>
> I am sorry that this topic does not 100% fit this group, but in lack of
> a better alternative I'll still give it a try. The problem I am
> describing is predominant in Linux/Unix/MacOSX, so perhaps you can help
> me out with some expertise.
>
> I am trying to emulate a FAT16 USB device on a microcontroller with
> ridiculously little RAM. Read and write accesses are directly forwarded
> to external chips such as EEPROMs. The FAT part is hardcoded.

I do not understassnd this. How can a FAT be hard coded?
Or do you mean you have a fixed and unalterable 'format'


So far, I
> have expected the location of the files to remain fixed - whenever the
> host writes to (or reads from) a specific sector, it would be mapped to
> the external memory. However, when it comes to writing to an existing
> file, it appears as if many OSes, instead of just overwriting the
> existing data, make a new file system entry, write the data to an unused
> cluster, and delete the old entry.

That is entirely possible, yes.

> Of course, then the data will be
> located at a different sector address, and my dumb little device will be
> confused. What I now want to try is to keep track of the OS' changes in
> the FAT structure

I dont understand how you expect to make a devicve that does not allow
the FAT to be written..


All you have to do is supply sector track read and write and let the OS
do what it does best. sort out which to use and when.


and expect the data to be written at the modified
> location. However, there are some cases where this is plain impossible,
> and I really need to know them in order to come up with an
> as-general-as-possible solution.
>

Its called using a track for the FAT. That's why its there.

> So here's my question to all you Linux experts out there: Making a new
> file should take three steps: allocate clusters in the FAT, enter file
> in the directory structure, write data. Can anyone help me with some
> information from the Linux kernel that would help me understand in which
> order this is done, e.g. when calling fopen() from C/C++?
>
> Thanks,
> Matthias
From: MatthiasH on 13 Jul 2010 07:11
The Natural Philosopher wrote:
>> The FAT part is hardcoded.
>
> I do not understassnd this. How can a FAT be hard coded?
> Or do you mean you have a fixed and unalterable 'format'

I have a writable root directory, and one writable sector in the
allocation table. Everything else is constant (program memory on my uC).

>> However, when it comes to writing to an
>> existing file, it appears as if many OSes, instead of just overwriting
>> the existing data, make a new file system entry, write the data to an
>> unused cluster, and delete the old entry.
>
> That is entirely possible, yes.

So I was wondering if there is any convention on the order of these
accesses, in particular for multithreading environments...

>> Of course, then the data will be located at a different sector
>> address, and my dumb little device will be confused. What I now want
>> to try is to keep track of the OS' changes in the FAT structure
>
> I dont understand how you expect to make a devicve that does not allow
> the FAT to be written..

(see above. FAT and root dir are writable)

> All you have to do is supply sector track read and write and let the OS
> do what it does best. sort out which to use and when.

Exactly. I have to find out which write accesses to forward to which
location in the nonvolatile memory. This is easy if the order of the
accesses is "rootdir first, then data" because I know where the data are
supposed to go. If the access is "data first", then I would have to
buffer the data before I get to know where to put them. Which I can't,
because I have almost no RAM.

> and expect the data to be written at the modified
>> location. However, there are some cases where this is plain
>> impossible, and I really need to know them in order to come up with an
>> as-general-as-possible solution.
>>
>
> Its called using a track for the FAT. That's why its there.

I'm sorry, but I don't think I get that last part.
From: The Natural Philosopher on 13 Jul 2010 08:15
MatthiasH wrote:
> The Natural Philosopher wrote:
>>> The FAT part is hardcoded.
>>
>> I do not understassnd this. How can a FAT be hard coded?
>> Or do you mean you have a fixed and unalterable 'format'
>
> I have a writable root directory, and one writable sector in the
> allocation table. Everything else is constant (program memory on my uC).
>
>>> However, when it comes to writing to an existing file, it appears as
>>> if many OSes, instead of just overwriting the existing data, make a
>>> new file system entry, write the data to an unused cluster, and
>>> delete the old entry.
>>
>> That is entirely possible, yes.
>
> So I was wondering if there is any convention on the order of these
> accesses, in particular for multithreading environments...
>

None that I would rely on..

>>> Of course, then the data will be located at a different sector
>>> address, and my dumb little device will be confused. What I now want
>>> to try is to keep track of the OS' changes in the FAT structure
>>
>> I dont understand how you expect to make a devicve that does not allow
>> the FAT to be written..
>
> (see above. FAT and root dir are writable)
>

>> All you have to do is supply sector track read and write and let the
>> OS do what it does best. sort out which to use and when.
>
> Exactly. I have to find out which write accesses to forward to which
> location in the nonvolatile memory. This is easy if the order of the
> accesses is "rootdir first, then data" because I know where the data are
> supposed to go. If the access is "data first", then I would have to
> buffer the data before I get to know where to put them. Which I can't,
> because I have almost no RAM.
>
>> and expect the data to be written at the modified
>>> location. However, there are some cases where this is plain
>>> impossible, and I really need to know them in order to come up with
>>> an as-general-as-possible solution.
>>>
>>
>> Its called using a track for the FAT. That's why its there.
>
> I'm sorry, but I don't think I get that last part.

Mutial confusion then.

Why cannot you just let the OS write sectors and the FAT as it wills?

I mean either you are trying to look just like a disk drive (in which
case all you need to do is read and write 'sectors'), or you are going
to have to write your own driver.

Or worse, your own OS..


From: MatthiasH on 13 Jul 2010 16:37
The Natural Philosopher wrote:

> Why cannot you just let the OS write sectors and the FAT as it wills?
>
> I mean either you are trying to look just like a disk drive (in which
> case all you need to do is read and write 'sectors'), or you are going
> to have to write your own driver.
>
> Or worse, your own OS..

The goal is that the device should work without special driver, under
any OS. My device wraps the contents of the external memory chip (a few
kb in size) into a single file in a FAT16 system which can be read
(works 100%) and written (works under Windows). Think of it as the EPROM
programmer anyone would have dreamt of in the 90s: no drivers, no
special software; access the chip through a file. For the reading part,
I'm already there. For the writing, there is the problem I described
earlier on. I need a way for my firmware to determine which cluster
currently holds the file contents, whenever the OS decides to change that.
 |  Next  |  Last
Pages: 1 2
Prev: How to define which user can "sudo" and for which commands ?
Next: single disk to raid 1