Following teachers’ return for this academic year in September, we suddenly found ourselves with a frequent issue. After waking the Surface Pro devices from sleep, the Type Cover would often not respond to keystrokes. The on screen keyboard was not affected, but USB keyboards also stopped working. The Type Cover trackpad would continue working fine.
A full Windows restart would always bring back the keyboard functionality.
This triggered a challenging investigation to determine what was wrong. The fact that we had made no significant software changes that should affect this over the summer made me look, with guidance from Microsoft Surface Business support, to Windows Updates as a possible issue. Rolling back both September and August’s Windows Updates did not seem to have any effect.
Clearly this wasn’t a wide enough issue to be affecting everyone, or many more customers would be up in arms about having to restart 6 or 7 times in a working day!
With the issue affecting a wide range of different Surface Pro devices and different Type Covers, it looked more likely to be a software issue than hardware. Predictably, perhaps, I was unable to reproduce the issue on a device with nothing but a stock Windows install on it… it’s got to be software. Right?
The build of Windows we run is kept as simple and close-to-stock as possible, for exactly the reason that it saves you from this type of issue! Of the software we do run, the prime suspects seemed to be:
- third-party endpoint security software
- Veyon Classroom Management software
I dug a little deeper into what Veyon brings along to do its magic. Its ability to remotely control other systems for classroom management purposes, including remotely inducing the Secure Attention Sequence (Ctrl-Alt-Delete to normal folks!) means that it must have some kind of driver installed that permits this functionality. Eventually, it dawned on me that this interacts with the keyboard, making it a good candidate for the culprit for, you know, keyboard problems.
Scouting for the .sys
I started by looking in the Veyon distribution for .sys files that would indicate kernel drivers. I figured — there must be a kernel driver running to do this functionality, and it must be in the Veyon distribution. Right??
I didn’t find anything, but there was an executable called install-interception.exe. I tracked this down to this GitHub project, where the API is open source, but the driver is not. I still couldn’t find the .sys file to dig into it and find out if its interaction with the keyboard and sleep/wake could be at fault. In fact, I couldn’t even find the name of the driver file this installer would drop.
I took to various tools including Pavel Yosifovich’s System Explorer to look at the Windows Object Manager namespaces and find the actual driver object in question.
A cursory glance revealed \Device\Interception00 – Interception19, with symlinks pointing at each of these also located in GLOBAL??. I wasn’t able to query the driver object to directly determine the .sys file that implemented these devices. A search on the C drive for interception didn’t give me anything useful either. Would the Interception devices not be implemented in a file called interception.sys? Apparently not.
Kernel drivers like these are registered in the same way as user-mode Windows system services, so I hopped over to the Registry to see if I could track down the .sys file. Nothing named interception or similar. It’s a little odd that this entirely legitimate driver (at least, Veyon’s use of it is entirely legitimate) is difficult to track down.
Gotta unpack ’em all
All I had was install-interception.exe which, after running and restarting, did cause the Interception00–19 drivers to exist. So the .sys must be packed inside this executable and it is unpacked when you install.
I fired up install-interception.exe in x32dbg and set a breakpoint on WriteFile. It must be writing a file to disk at some point with the driver.
After some missteps and corrections in setting the breakpoint correctly, I ended up here. The arguments to WriteFile appear on the stack, including the destination path: C:\WINDOWS\system32\drivers\keyboard.sys.
I wouldn’t have guessed that this path, a built-in path, with the innocuous name keyboard.sys, was to do with a third-party driver. I’m not super familiar with where kernel-mode software drivers would normally live, but again, it’s almost like Interception is trying to be difficult to identify on a system.
The file is, however, signed by the author of the project. Once you know what you’re looking for, it makes some sense I suppose.
Could we have found it in the Registry?
Let’s spin back a bit and head back to the Registry. Now that we know it’s keyboard.sys (and mouse.sys for what it’s worth), would we have been able to find the path to the .sys file in the Registry?
We go to HKLM\SYSTEM\CurrentControlSet\Services\keyboard and find that, unlike most of its comrades in the Services key, keyboard doesn’t have an ImagePath value. My guess is that in these cases, Windows infers the path to the .sys from the key name and looks in System32.
Shifting into reverse
Now we actually have a binary, I can put my limited, but developing, reverse engineering skills to use to find out how likely it is that keyboard.sys is the culprit of our issues here. I figured that we’d be looking for interactions with the power management subsystem. If I saw these, I felt we could be pretty confident this was the source of the issue, although, spoiler alert, I don’t exactly know why, or why these issues had not manifested themselves prior to August.
Bringing the driver into Ghidra, examining the entry point, and digging through some functions it calls, quickly revealed this correlation:
I am not presently skilled or knowledgeable enough to assert why Interception is the problem is this sleep/wake scenario with Surface Type Covers, but I hypothesise the following:
- keyboard.sys is a driver in the keyboard input filter driver stack.
- The ‘removal’ and ‘insertion’ of a new keyboard device requires some interaction with keyboard.sys in the driver stack.
- The Type Cover is ‘removed’ when the Surface Pro goes to sleep (especially when using the Type Cover to put the device to sleep by closing it against the display).
- Additionally, the power management subsystem is obviously involved in sleep/wake.
- The Type Cover doesn’t immediately resume and reattach after waking from sleep (this is observable if you tap Caps Lock and see when the light on the Caps Lock key responds. Compare with tapping Function and when this light responds — as soon as the Type Cover is powered.)
- There is some kind of race condition between the power management involved code in keyboard.sys and its handling of whether keyboard input devices in the driver stack are enabled for input.
(Subsequently, I found a better explanation.)
As it happens, we don’t need the Interception driver on teachers’ devices — they are doing the controlling, and pupil computers are the ones being controlled, where the driver may be useful.
So, removing the Interception driver resolved the issue. You can run interception\uninstall.bat from within the Veyon distribution to achieve this. A restart is needed.
It’s also possible to install Veyon and pass a switch /NoInterception to avoid installing the driver, which I will now do on new installs!
This is possibly the issue #25 on the Interception project, but regrettably I can’t add my voice to that issue, as it is locked.
I will flag it with the Veyon project too. It is an edge case to some extent — this only manifested on Surface Type Covers.
This was a good, deep problem that I enjoyed diving into. It was not so enjoyable to feel, at points, helpless while my users had a frustrating experience with their keyboards (it took a long time of trial and error to get to this conclusion), but I am glad to have finally figured out how to work around it.