Crashes and recovery
How a failing driver is contained and restarted.
In DurgX a crash is expected, contained and handled. It is not an emergency.
What happens when a driver crashes
- The driver does something illegal, such as touching memory it does not own.
- The kernel catches it. Only that one program is stopped. The kernel does not crash.
- The kernel sends a fault report to the driver's supervisor: a program that was registered to look after it. The report says which program failed, why, and where.
- The supervisor decides what to do.
What a supervisor can decide
| Choice | What happens |
|---|---|
| Restart | Start a fresh copy, give it the same permissions back, and replay any unfinished work. |
| Switch off | Remove its permissions, mark the device offline, and tell the programs that used it. |
| Escalate | If it keeps failing within a short time, stop retrying and raise the problem to a higher level. |
This works like supervisors in the Erlang language, with one difference: each unit is separated from the others by hardware, not just by convention.
What this means for your design
- Write drivers so they can be restarted safely. See Writing a driver.
- Keep important state outside the driver, for example in a queue that survives a restart.
- Choose your restart limits per device. A sensor might restart forever, a storage driver might not.
Note
You can try this on the home page in Try it: crash a driver.