Why Operating Systems Programming is so difficult?

The character of operating system programming has remained more or less constant over the years. That is a polite way of saying that it is still as scary as ever. Why it is so? Complex system requires complex programming methods and this goes against the spirit of the dictum "Good programs are easier to understand and develop". So can we conclude that system programming being done is bad programming. Or the programmers are not following best practices and all the elaborate methods created to ease programming. Or they are still dealing with a lot of legacy code. Looking at the output "they" are the best talent. Probably its the fact that they are good that they create something so brilliant that nobody can fathom or appreciate. The reality needs to be explored.

System Programming
So what exactly is operating system programming. Is it really something different from regular programming or its just a subset of programming. System Programming in contrast to application programming is directed towards hardware/system programming while Application Programming deals with creating software which interact with the user.

Sources of Complexity
The complexity in the dealing with systems programming language amongst others is due to the following reasons.

1. Legacy
2. Diversity

Legacy
Legacy code is very hard to get away with in application programming but when it comes to Systems Programming. Legacy code is EXTREMELY hard to get away. Since the legacy code has been written with an extremely large investment of time and also due to scarcity of expertise available so it is more feasible to use legacy code instead of re-inventing the wheel.

Diversity of Hardware
This is the other problem which leads to complicated code. Hardware diversity needs to be catered by system programmers as compared to application programmers who have the luxury of using interfaces (provided by system programmers) to hardware.

How to get over it

There is no short cut to success in the system programming domain. One has to go through the rigor of learning the issue. An easier way could be using the input-output model. Input output model is a way of getting reducing the code's complexity and focusing on inputs and outputs of the given algorithm/ function. This helps in first understanding the basic goals before diving deeper into the code. 

Another familiar method is to develop state diagrams. This methodology requires a little more skill than the previous method but can be extremely helpful in figuring out complicated code.