Various comments from reading the code

Dec 14, 2008 at 6:32 PM
I don't have Windows at home anymore so I'm limited w.r.t actually installing and trying Singularity. However I read some of the code and have the following remarks:
  • There's a surprising amount of XML parsing going on inside the kernel, and it's direct DOM manipulation. Typically if the format is wrong either that entry is ignored, or it silently does the wrong thing, or the kernel DebugBreaks. Have you considered using something a bit more strongly typed, like protocol buffers or even some home-brewed system?
  • Is there a style guide for contracts? The HttpRequestContract surprised me because it looks amazingly heavy .... surely this should be a rep struct and not a list of GetFoo messages? Why does static data need a contract of its own?
  • Should the Singularity style guide specify String vs string? I saw both in use and as a C# neophyte it was a bit confusing.
  • There seem to be several logging frameworks in use simultaneously. I read the ETS design note (btw - propz for including all of those), but I'm not sure where it's actually used. ETS seems like an efficient framework in general, is it only for OS services or applications too? Most of the code I saw was a mish-mash of different styles, with #ifdef used a lot.
  • What version of C# is Sing# based on? I was surprised to see a lack of generics in things like SortedLists inside the IO system.
  • The TPM integration is an interesting area for research. I wonder if it could be used to implement an undebuggable/uncopyable program such that it decrypts itself only if the TPM indicates that a trusted kernel version is running and that no debugger is attached. Presumably you'd need to encrypt the bulk of the program, and have a stub loader that generates the key from the PCR contents somehow. Or alternatively, the stub program attests the system state to a remote server (over SSL) which coughs up the key only if the attested state proves there is no debugger running (and thus, the key is safe inside the program). What is the current state of research into this topic?
Thanks! I'll be back with more questions later :)
Coordinator
Jan 3, 2009 at 12:11 AM
Hi cantarell, I'm glad you've taken an interest in reading the codebase, and I apologize for my slow response. Some responses:

> There's a surprising amount of XML parsing going on inside the kernel, and it's direct DOM manipulation. Typically if the format is wrong either that entry is ignored, or it silently does the wrong thing, or the kernel DebugBreaks. Have you considered using something a bit more strongly typed, like protocol buffers or even some home-brewed system?

Most of the XML processing happens in the context of processing manifests; there's no particular reason that manifests are written in XML, other than that this makes them easy for humans to review. Although the parsing should be robust against incorrect manifests, it can produce a fatal error in this case, because any manifest correctly generated by the build should parse successfully. It might be useful to have an application in another SIP responsible for parsing and verifying manifests, producing an in-memory strongly-typed structure for the kernel to consume, but this isn't something we've pursued yet.

> Is there a style guide for contracts? The HttpRequestContract surprised me because it looks amazingly heavy .... surely this should be a rep struct and not a list of GetFoo messages? Why does static data need a contract of its own?



> Should the Singularity style guide specify String vs string? I saw both in use and as a C# neophyte it was a bit confusing.

Yes, it should - it would be a good idea (and a straightforward task) to go through and ensure consistent use. For integer types, there's a tendency to use the full type names in areas where the exact size is important, since they specify the number of bits in the name; but this distinction is unimportant for strings.

> There seem to be several logging frameworks in use simultaneously. I read the ETS design note (btw - propz for including all of those), but I'm not sure where it's actually used. ETS seems like an efficient framework in general, is it only for OS services or applications too? Most of the code I saw was a mish-mash of different styles, with #ifdef used a lot.

You're right, there was an older logging framework and a new and more sophisticated one added more recently (ETS). I don't know enough about ETS to tell you where it can be used, but there are a number of places where the code can use it and hasn't been refactored to do so, since it's a relatively recent addition.

> What version of C# is Sing# based on? I was surprised to see a lack of generics in things like SortedLists inside the IO system.

Because the Bartok MSIL-to-native compiler was developed concurrently with Singularity, it was constrained to use .NET Framework 1.1 features up until a couple years ago. A lot of this code is still left over and hasn't been refactored to use generics.

> The TPM integration is an interesting area for research. I wonder if it could be used to implement an undebuggable/uncopyable program such that it decrypts itself only if the TPM indicates that a trusted kernel version is running and that no debugger is attached. Presumably you'd need to encrypt the bulk of the program, and have a stub loader that generates the key from the PCR contents somehow. Or alternatively, the stub program attests the system state to a remote server (over SSL) which coughs up the key only if the attested state proves there is no debugger running (and thus, the key is safe inside the program). What is the current state of research into this topic?

As with any DRM enforcement measure, it's ultimately just making the attack more expensive; for example, this approach would not be immune against virtual machine attacks where the system is booted inside a virtual machine and then its state is dumped. It would also be susceptible to cold boot attacks. Nevertheless if you're interested in this I think it would be an interesting idea to pursue. So far the TPM support in Singularity has only been used to do basic TPM research; there's no public information related to it other than the information in the RDK.

I hope this helps! Please ask if you have any other questions, and I look forward to seeing what you'll do with Singularity.
--
Derrick Coetzee
Microsoft Research Operating Systems Group developer
Coordinator
Jan 3, 2009 at 12:14 AM
Oops, I forgot to answer a question.

> Is there a style guide for contracts? The HttpRequestContract surprised me because it looks amazingly heavy .... surely this should be a rep struct and not a list of GetFoo messages? Why does static data need a contract of its own?

No, there's not currently any style guide for contracts, other than looking at existing contracts. Some contracts, like the HttpRequestContract, could be refined to be more efficient, but the general philosophy in Singularity is that messages are cheap, so it's alright to send more than you have to. This can change in scenarios where the channel is crossing a hardware domain or for some other reason has longer latency than usual.

I hope this helps.
--
Derrick Coetzee
Microsoft Research Operating Systems Group developer
Jan 3, 2009 at 2:44 PM
Thanks Derrick, those answers are helpful.

I agree that TPMs don't protect against hardware attacks today, although the should protect against running inside a VM. So that makes them not so useful for real DRM, but they can still be helpful for avoidance of online game cheating where most users are not going to become hardware hackers.

Something that interests me is how to make the robustness offered by a micro-kernel system like Singularity "real", that is, actually robust. For instance in theory Linux should be more robust than Windows because the graphics/windowing subsystem runs as a user-mode process, and can easily be restarted if it crashes. But in practice if the X server dies everything dies, because most apps aren't able to re-establish their state with the other side. The few apps that can have X specific implementations and still abort if other system servers die.

Singularity would presumably have that problem in extremis, eg if a file system SIP crashes then every other SIP crashes in a domino effect as their file handles are invalidated.

Solving it I guess would need some kind of global checkpointing, such that each SIP is checkpointed at known good times by the system, and if it crashes that SIP is rolled back, with every other SIP depending on it also rolled back to a known good state. How this fits with the whole channel/contract design, I don't know. It may not even be worth persuing, as the whole microkernel/SIP design has benefits beyond robustness anyway.
Feb 3, 2009 at 11:13 PM
That may I say suck if it happened ,
What is the point of this project if it cant even establish some mean protection its like building domino.
You never know which part of the whole could collapse to give way to the others if a mistake is made.
But then again when did a file system crashed anyway if so what is the appropriate action.
Cant imagine write something to continuously check he I'm alright go right ahead. Or is the channel contract design
the fail safe towards that.    
Coordinator
Mar 20, 2009 at 12:37 AM
Hi cantarell, sorry for the slow response. That is an interesting point and a valid avenue for further research and development. It is a practical issue in development that rare error conditions are frequently dealt with poorly - thus the erratic behavior of a system when running out of memory or disk space - and this is not a problem that goes away with the use of SIPs. I think part of the way this can be addressed is via small intermediary SIPs that handle restarting failed SIPs and temporarily buffer requests while they're down (this would help solve the file system example). But it's not clear that a generic solution exists, since we don't want contracts to buffer arbitrary amounts of data and take up unbounded memory. If you have ideas for this you'd like to discuss please feel free, and thanks again for your interest!
--
Derrick Coetzee
Microsoft Research Operating Systems developer