status update: x86 interpreter...
in [ASM]
|
Prev: LPC2478 arm7tdmi 56mhz 8kb cache 32mb ram, jpeg decoding performance
Next: LPC2478 arm7tdmi 56mhz 8kb cache 32mb ram, jpeg decoding performance
From: BGB / cr88192 on 9 Nov 2009 14:23 seems I had not been telling people here about this, so, I just figured I would give a status update... x86 ISA, still "mostly" supported: x87 has not been well tested (and even then, does not currently support "subtle issues" such as rounding modes, ...); SSE/SSE2, still not gotten to finishing up support as of yet (mostly lacking is packed integer support, ...). the x86 core ISA thus far seems to work in tests. performance: currently about 13 MIPS in tests (testing a simple loop doing lots of memory IO and bit-twiddling); in terms of time, the interpreter is about 76x slower than native (vs the same loop compiled in native code). the interpreter is currently pure C, and is not particularly micro-optimized, so this factor could still be improved some (ASM and/or JIT being options, but at the moment I am not considering them, since as I see it, having the thing work acceptably is more important than speed at present). similarly, I have currently also excluded optimizations which would unecessarily mess up my current design practices (such as violating modularity, lead to overly large amounts of redundancy or special cases, ....). functionality: I have a C library on the thing (PDPCLIB), and the code making use of said C library seems to start up and work ok. effort is currently underway to be able to support JNI as well (I am using JNI to interface with my native object system, although it is not an exact match with the JVM object system). note that, in this case, I am generally importing APIs via a JNI-style "structs of function pointers" strategy (although this is rather inconvinient and tedious). my main reason for doing so is to avoid needing physical linkage with other parts of my framework, and to more easily allowing falling back to native facilities (or automatically disabling stuff), should stuff not be available. internally, some facilities are being marshalled into the interpreter via JNI and JVMTI, and I am implementing similarly designed interfaces for other pieces of functionality (VFS, ASM / native-code reflection, ...). (note that my framework is NOT a JVM, even if I am using JNI and JVMTI to marshall certain facilities, ...). I am currently also working some on POSIX functionality (within the virtualized world). I don't expect a "complete" implementation, but hopefully enough for my uses. currently supported POSIX features: basic file IO, dlopen/dlsym, ... still lacking: more advanced file features (stat, readdir, ...), sockets, pthreads, ... I am making use of a posix-style "process" model (AKA: multiple running processes, each with a PID, assigned UID and GID, ...). I am considering using UID and GID for the security model, but may "re-interpret" how they work (nevermind lacking support in my framework's VFS). "POSIX shell" support is nebulous (some "shell" related functionality has been written, but I am not sure what here is actually relevant or what all is worth trying to implement). shells may be attached to processes, but currently are not themselves processes (IOW: they exist as native code). I am still using PE/COFF EXE's and DLL's (but may consider leaving off the '.EXE' extension, or allowing use of '.SO' for DLL's). little beyond inconvinience (having the means to compile code as ELF on Windows, or writing a loader) prevents using ELF. I "could" take a "flex" strategy and use whatever is more convinient (vs forcing Linux builds to use PE/COFF), although it can be noted that they are not strictly equivalent (and could also pose issues for dumb build tools). (technically, mostly all this is more a matter of aesthetics and available compilers...). for technical reasons, I may impose limits as to how much memory can be used by virtual processes (otherwise, I would need a "proper" MMU and support for swapping in order to work effectively on 32-bit hosts). or such...
From: Rod Pemberton on 10 Nov 2009 10:13 "BGB / cr88192" <cr88192(a)hotmail.com> wrote in message news:hd9q87$6cp$1(a)news.albasani.net... > > x86 ISA, still "mostly" supported: .... > x87 has not been well tested (and even then, does not currently support > "subtle issues" such as rounding modes, ...); Is this really needed? What x86 code are you running through your emulator? BIOS? Video BIOS? > SSE/SSE2, still not gotten to finishing up support as of yet (mostly lacking > is packed integer support, ...). Ditto. > currently supported POSIX features: basic file IO, dlopen/dlsym, ... > still lacking: more advanced file features (stat, readdir, ...), sockets, > pthreads, ... Doug Gwyn's PD libndir has readdir, opendir, seekdir, etc. directory functions. It's usually name libndir.tar.z or libndir-posix.tar.z The two versions are slightly different. There is also posix-1.0-src-11.00.tar.gz which has more files. I had to locate a new link for them. This site has them: http://ftp.lahtermaher.org/pub/unix-c/languages/c/ posix-1.0-src-11.00.tar.gz: http://hpux.connect.org.uk/ftp/hpux/Languages/posix-1.0/ Rod Pemberton
From: BGB / cr88192 on 10 Nov 2009 11:28 "Rod Pemberton" <do_not_have(a)nohavenot.cmm> wrote in message news:hdc024$939$1(a)aioe.org... > "BGB / cr88192" <cr88192(a)hotmail.com> wrote in message > news:hd9q87$6cp$1(a)news.albasani.net... >> >> x86 ISA, still "mostly" supported: > ... > >> x87 has not been well tested (and even then, does not currently support >> "subtle issues" such as rounding modes, ...); > > Is this really needed? > > What x86 code are you running through your emulator? BIOS? Video BIOS? > mostly userspace code, since it is essentially interpreting/emulating C code compiled into a form usable from the simulated userspace. it currently does not (completely) simulate either Ring-0 features, or the hardware. like a traditional userspace, it is assumed that all communication with the "kernel" (in this case, the interpreter itself, and everything outside said interpreter) is via system calls (since, as can be noted, traditional userspace apps don't have any access to the HW anyways, only the flat 4GB address space, of which I am currently dedicating 2GB as usable for the app, 1GB as "shared", and 1GB for internal interpreter use, mostly for swizzling handles between address spaces, ...). FWIW, I could give a simulated app up to 3.5GB of address space (essentially giving up the idea of a "common region" and limiting the raw number of references which may be swizzled), but this is not needed, and OS's such as Windows typically impose a 2GB app address-space limit anyways... I may also consider address-space randomization as well, but have not done so as of yet... as for x87: typically, it is locked into full 80-bit mode, and the low order bits are subject to "whatever" roundoff. some operations only have 64 bit accuracy (currently, mostly for things like sin/cos/...). >> SSE/SSE2, still not gotten to finishing up support as of yet (mostly > lacking >> is packed integer support, ...). > > Ditto. > yep. I also have a few "bits and pieces" of MMX functionality, but this is not a particularly high priority (since, I figure, almost nothing really uses MMX anyways...). >> currently supported POSIX features: basic file IO, dlopen/dlsym, ... >> still lacking: more advanced file features (stat, readdir, ...), sockets, >> pthreads, ... > > Doug Gwyn's PD libndir has readdir, opendir, seekdir, etc. directory > functions. It's usually name libndir.tar.z or libndir-posix.tar.z The > two > versions are slightly different. There is also posix-1.0-src-11.00.tar.gz > which has more files. > > I had to locate a new link for them. This site has them: > http://ftp.lahtermaher.org/pub/unix-c/languages/c/ > > posix-1.0-src-11.00.tar.gz: > http://hpux.connect.org.uk/ftp/hpux/Languages/posix-1.0/ > well, what is to say they are compatible with my implementation?... (well, even if not entirely, maybe the headers and some of the code may be useful, and I can hack on the rest...). I may look, but thus far most operations of this sort are simply wrappers around system calls, with the actual machinery taking place in the "host app". the main exception is the C library, since it does a lot of things which can't be directly provided via system calls, and allows marshalling a much more complex interface (the C90 runtime) through a simpler interface (a relatively smaller number of basic system calls). the main reason I don't have readdir already is because the interpreter is connected to the VFS via an older interface struct which does not provide the needed API calls (basically, I am using the same interface which was used for my assembler, which only provided for basic file IO), and I am in the process of designing a new VFS interface struct (which may also provide for sockets), but this has not yet been finished. this would be followed by re-routing the traffic via the new interface, and maybe implementing the needed syscalls (in the interpreter, and also in the 'vxcore' DLL). note: 'vxcore.dll' essentially serves a similar role to 'kernel32.dll' and 'ntdll.dll' in Windows (AKA: mostly raw system calls and wrappers and similar). for example, I emulate some calls via system calls which are often following a mix of POSIX and Win32 conventions. for example, the libdl is faked via syscalls which fake, for example, 'LoadLibrary' and 'GetProcAddress', and others via a faked 'VirtualAlloc', .... ('mmap' would be a wrapper over 'VirtualAlloc' and 'CreateFileMapping' and similar...). this is partly because, at the basic level, I am implementing syscalls using whichever style is easiest to emulate... similarly, there is a big chunk of syscall space currently dedicated to JNI (I am marshalling the calls, so both struct-based JNI, and a more direct interface to the host object system, may be provided via this means). an awkwardness though is that x86 is not JBC, and so there is no good way at present to handle class and interface definition in 'vitrual' code ('classloader' doesn't really work, and the facilities provided by 'dyClass' don't route through JNI, ...). (I may instead resort to a trick involving statically defining data via statically-initialized structs and strings and using special exports to describe it...). or such... > > Rod Pemberton > >
From: Rod Pemberton on 10 Nov 2009 17:56 "BGB / cr88192" <cr88192(a)hotmail.com> wrote in message news:hdc5b7$p2m$1(a)news.albasani.net... > > [POSIX support code] > > well, what is to say they are compatible with my implementation?... > Nothing... But, you mentioned PDPCLIB somewhere. I don't think it has them either, does it? Rod Pemberton
From: BGB / cr88192 on 10 Nov 2009 18:37
"Rod Pemberton" <do_not_have(a)nohavenot.cmm> wrote in message news:hdcr5j$dis$1(a)aioe.org... > "BGB / cr88192" <cr88192(a)hotmail.com> wrote in message > news:hdc5b7$p2m$1(a)news.albasani.net... >> >> [POSIX support code] >> >> well, what is to say they are compatible with my implementation?... >> > > Nothing... But, you mentioned PDPCLIB somewhere. I don't think it has > them > either, does it? > nope, but it is not strictly a thin wrapper either. PDPCLIB essentially wraps a small number of syscalls, and exports a much more complex API. much of POSIX, however, would seem to be things which would be directly implemented via syscalls. but, as I said earlier, I would look into this, and see if a lot can be reused, I have just not been able to today as I had been off at classes and stuff... and, worst case, it will probably at least provide some usable headers (for which I can fill in the backend logic, ...). so, more status for later. or such... > > Rod Pemberton > > |