I ran into some trouble recently with a machine that had previously been registered for Intune/Microsoft Endpoint Manager AutoPilot deployment hanging on a Windows reinstall (the SSD had been replaced).

The machine would sit at “checking connection to Microsoft. This might take a while”.

Take a while it did — a spell overnight on this very screen would not help. I used Shift+F10 to get some diagnostics tooling on the system. I could see references to a /join HTTPS endpoint being accessed that seemed to be Intune-related, but it was neither obviously succeeding nor failing.

Some perusal of logfiles suggested to me that UEFI variables are involved in the AutoPilot process.

Fortunately, the machines in question are desktop PCs. A very simple way to (destructively!) clear out UEFI variables was to remove the CMOS battery for a period of time. Upon trying again, we jump right past “checking connection to Microsoft” and can move forward with the install.

On systems where a battery pull is not effective, it may be worth getting yourself into a UEFI shell and using dmpstore to identify UEFI variables in NVRAM that may be related to AutoPilot and deleting them. Sorry I can’t be more specific!

Today’s malware-loader-du-jour, Bumblebee, has been seen achieving initial access through phishing sites that convince a user to mount a downloaded ISO image. This may be a reaction to Microsoft’s recent improvements to macro-enabled document security.

Adversaries push ISO files through compromised email (reply) chains, known as thread hijacked emails, to deploy the Bumblebee loader. ISO files contain a byte-to-byte copy of low-level data stored on a disk. The malicious ISO files are delivered through Google Cloud links or password protected zip folders. The ISO files contain a hidden DLL with random names and an LNK file. DLL (Dynamic Link Library) is a library that contains codes and data which can be used by more than one program at a time. LNK is a filename extension in Microsoft Windows for shortcuts to local files.

https://cloudsek.com/technical-analysis-of-bumblebee-malware-loader/

One of the things that we can do to help our users avoid this new initial execution foothold is by blocking the mounting of ISO images, as long as you can be confident this will not break anything they actually need to do! I am fortunate enough to be able to do this.

(Djordje Atlialp shows us how to achieve this with classic GPOs, and also a more comprehensive neutering of ISO files.)

Here is what I have rolled out as an Intune PowerShell Script to block the mounting of ISOs. No reboot is required. Users will see the Mount option disappear from the context menu of an ISO file within File Explorer and will be unable to double-click to mount a malicious ISO. Or, indeed, any ISO. 😉

We will head to Microsoft Endpoint Manager admin center, go to Devices > Scripts and create a new Windows 10 and later PowerShell script.

The Intune Script

UPDATE: I have made some improvements — namely, the previous one liner will cause failures to be reported in Intune on subsequent runs. We will now only add the value where it does not exist, and we will add support for Windows.VhdFile as well. It’s no longer a one-liner!

$items = @(
    @{
        path = "HKLM:\SOFTWARE\Classes\Windows.IsoFile\shell\mount"
        valueName = "ProgrammaticAccessOnly"   
    }
    @{
        path = "HKLM:\SOFTWARE\Classes\Windows.VhdFile\shell\mount"
        valueName = "ProgrammaticAccessOnly"
    }
)

foreach($item in $items) {
    if ($null -eq (Get-Item -Path $item.path).GetValue($item.valueName)) {
        New-ItemProperty -Path $item.path -Name $item.valueName -Value ""
    }
}

The body of the script can be as follows:

New-ItemProperty -Path "HKLM:\SOFTWARE\Classes\Windows.IsoFile\shell\mount" -Name ProgrammaticAccessOnly -Value ""

(This REG_SZ value need only exist, with a blank string as its Data, for this to work.)

Assign it to the device group and you are all set.

Removal

To undo this change, we can reverse what we’re doing and Remove-ItemProperty on the items we added:

$items = @(
    @{
        path = "HKLM:\SOFTWARE\Classes\Windows.IsoFile\shell\mount"
        valueName = "ProgrammaticAccessOnly"   
    }
    @{
        path = "HKLM:\SOFTWARE\Classes\Windows.VhdFile\shell\mount"
        valueName = "ProgrammaticAccessOnly"
    }
)

foreach($item in $items) {
    if ($null -ne (Get-Item -Path $item.path).GetValue($item.valueName)) {
        Remove-ItemProperty -Path $item.path -Name $item.valueName
    }
}

Conclusion

This doesn’t make you bulletproof, but will, if tolerated by your users, provide a substantial degree of protection, at the time of writing, from any number of current malware loaders that are using the ISO image technique to achieve initial code execution. The nature of the separate filesystem within the ISO presently prevents it from being marked as being from the Wild Wild West World Wide Web.

In a continuation of my desire to write really lightweight software that doesn’t add to the undesirable background bloat running on computers, I set about in June-ish to write something to improve upon a VBScript-Scheduled-Task-and-shutdown.exe gaffer tape of a solution to forcing a full shutdown when a computer is idle that I had previously cobbled together.

Power management in Windows is mature and capable, for sure, but what is less obvious is how to, on shared fixed desktop computers, actually trigger a proper shutdown and not just put idle machines to sleep. Hibernation is an option, of course, but the relentless increase in complexity of Windows brings to mind the other, stability-related, benefits of regular proper restarts.

So, then, we want something that:

• identifies when no-one is interactively signed in
• waits a configurable amount of time
• if still no-one has signed in in that time, shut down properly

Additionally, because this unavoidably must run with high permissions and regularly assess signed in users in the background, it should be a Windows service that is as lightweight and simple as possible. Reduced resource usage (RAM, CPU time in background) so we can shut down and have reduced resource usage (of electrical power). I can see the beauty of it already!

So I wrote ShutdownBuddy.

It is configurable through the registry:

HKLM\SOFTWARE\upfold.org.uk\ShutdownBuddy

EvaluationIntervalSeconds — DWORD. How frequently, in seconds, to evaluate for interactive sessions.

ShutdownAfterIdleForSeconds — DWORD. How many seconds of idle computer (i.e. no interactive sessions) before issuing a shutdown. This is periodically evaluated as above.

Like all my lightweight, C(++) Win32 projects, it is officially experimental as I am using these projects to learn how to write this kind of code properly. Any suggestions and improvements are gratefully received.

Despite the inexorable march towards running all workloads in the cloud, I see some specific advantages in maintaining some on-premises servers where this makes sense. Especially in a small scale environment like the one I am responsible for, this lets me do Interesting Things with my skill set and at small scale, only because I have full control over on-prem kit.

One of the options for backing up Hyper-V workloads I am looking at is Microsoft Azure Backup Server (MABS) v3. On a brand new deployment (Windows Server 2019), following Microsoft’s guide, I ran into issues with the MABS installer.

At the point of connecting to the Vault above, it would take a long time to “validate credentials” and then:

Invalid vault credentials provided. The file is either corrupt or does not have the latest credentials associated with recovery service. ID 34513

I found a way to work around this — we need to slipstream an updated version of the Microsoft Azure Recovery Service (MARS) Agent into the MABS installer before we run it.

Tidy Up First

If you have a messy server after some failed install attempts, roll back the VM if possible, or uninstall all MABS and MARS components with Add/Remove Programs.

Also, use the MMC certificates snap-in in Local Computer mode to remove any stale vault credential certificates.

To do, this launch mmc.

We will Add/Remove Snap-ins, and choose Certificates, Local Computer.

Remove any CB_ certificates from previous runs to avoid any possibility of confusion when the registration process runs again.

Take care to ensure you understand what you are doing before deleting certificates and keys. Do not delete anything you are not certain is unused and related to MABS. I cannot take responsibility for your loss if you follow this guide and have issues.

In the Azure Portal, go to your backup vault and check Backup Management Servers and Protected Servers, removing any registrations from previous failed installs, so we are installing into the vault with a clean state. Alternatively, create a new vault. I unfortunately ended up with my MABS server in Protected Servers and had to delete it. Ultimately it will be registered as a Backup Management Server, not a Protected Server.

The Workaround — Slipstream Updated MARS into MABS Installer

We are following Microsoft’s documentation on installing MABS, diverging only to slipstream in an updated MARS installer.

Download MABS from https://www.microsoft.com/en-us/download/details.aspx?id=57520

We will have the .exe and 7 .bin files. Run the .exe to extract the bin files. The install “media” in this case has now been extracted to C:\System_Center_Microsoft_Azure_Backup_Server_v3\System Center Microsoft Azure Backup Server v3.

Before we run the MABS setup wizard, we need to slipstream in the updated MARSAgentInstaller. Do not run the MARSAgentInstaller separately – we must have it invoked by the MABS wizard at the right time and in the correct context. If MARS is already installed when we try to install MABS, it will be unhappy and refuse.

Download the updated version of the MARSAgentInstaller https://download.microsoft.com/download/b/7/b/b7b5d2c9-d345-4d96-b48a-fe98a50f4c11/MARSAgentInstaller.exe

Copy and replace the version in C:\System_Center_Microsoft_Azure_Backup_Server_v3\System Center Microsoft Azure Backup Server v3\MARSAgent\MARSAgentInstaller.exe before running either setup program.

Now run C:\System_Center_Microsoft_Azure_Backup_Server_v3\System Center Microsoft Azure Backup Server v3\Setup.exe (the MABS installer, not the MARSAgent installer) and proceed as normal.

Provide the vault credentials we downloaded from the Azure portal to register the server as normal.

References

This references issue raised on TechNet forums. I cannot reply as the thread is locked.

https://social.technet.microsoft.com/Forums/en-US/bdcd9f72-ccbd-4b40-b984-5f3c09cf71f6/invalid-vault-credentials-provided

I found myself scratching my head on a new deployment of the Elastic Winlogbeat client on Windows, where the intent was to forward Event Logs to Graylog.

In the graylog-server.log file, I saw:

ERROR [AbstractTcpTransport] Error in Input [Beats/...] (...) (cause io.netty.handler.codec.DecoderException: java.lang.IllegalStateException: Unknown beats protocol version: 71)
ERROR [AbstractTcpTransport] Error in Input [Beats/...] (...) (cause io.netty.handler.codec.DecoderException: java.lang.IllegalStateException: Unknown beats protocol version: 69)

I turned off TLS on the client and the receiving Graylog Input, thinking it might be some TLS-related issue, to the same error.

To cut a long story short, decimal 71 and decimal 69 are ASCII codes for capital G and capital E — the first two bytes of an HTTP request that the Winlogbeat client was making to the Graylog input. Clearly the input is not expecting to receive ASCII “GE” to start the request!

It turns out the Graylog beats input desires the logstash format, not elasticsearch. I had been ignorant in just modifying the example config in the output.elasticsearch section, when this is not what Graylog wants.

I commented out the entire output.elasticsearch section and moved that configuration (hosts and the ssl options) into an output.logstash YML node.

output.logstash:
   hosts: ["x.y.z.aa:5044"]

This was unclear enough to me that I thought connecting the error messages above with this solution may prove useful for someone else who has the issue.

With a relatively recent Azure AD hybrid directory under our belts, we decided at work not to use the older Azure AD Connect tool and instead use the newer Azure AD Connect Cloud Sync. It’s lighter weight, doesn’t require a SQL database — lots of reasons to love it.

It does appear that, juuuust in time for our deployment, password writeback is supported, in preview.

However, I struggled to follow the official instructions to enable it, as the cmdlet did not seem to be available after importing the DLL.

Import-Module 'C:\\Program Files\\Microsoft Azure AD Connect Provisioning Agent\\Microsoft.CloudSync.Powershell.dll' 
Set-AADCloudSyncPasswordWritebackConfiguration -Enable $true -Credential $(Get-Credential)

Set-AADCloudSyncPasswordWritebackConfiguration : The term 'Set-AADCloudSyncPasswordWritebackConfiguration' is not
recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if
a path was included, verify that the path is correct and try again.
At line:1 char:1
+ Set-AADCloudSyncPasswordWritebackConfiguration
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (Set-AADCloudSyn...ckConfiguration:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

A little bit of investigating revealed that the DLL does indeed export the cmdlet, so what is going on?

I tried to run the above in PowerShell 7. It imported the cmdlet, but hit an issue with running it when importing its required libraries.

My Workaround

My workaround is to use PowerShell 7, but import the module with the -UseWindowsPowershell compatibility switch.

Import-Module "C:\Program Files\Microsoft Azure AD Connect Provisioning Agent\Microsoft.CloudSync.Powershell.dll" -UseWindowsPowerShell 
Set-AADCloudSyncPasswordWritebackConfiguration -Enable $true -Credential $(Get-Credential) 

And with that, password writeback is working on Azure AD Cloud Sync!

I have been getting back into the swing of building things just for fun and for exploration. I think a pressure can emerge that the things we create have to mean something, have to hit some mark of quality to be worthy of pursuing at all. I think that is a mistake, especially in that it discourages you from being open to areas where you don’t already have great expertise. So, here I am, putting into practice rejecting that pressure.

Continuing my theme of trying to work with lower-level APIs and with unmanaged languages like C and C++, I have developed a very simple client for Windows’ Volume Shadow Copy service, that allows me to create a shadow copy of a volume, copy some files in a folder to a destination, and then release the shadow copy.

I can see a use for this as part of a very low-tech backup solution where a drive is (most of the time) physically disconnected, and where you want to keep the technology stack as simple as humanly possible for the greatest flexibility in disaster recovery scenarios — BitLocker for external drives (compatible even with Windows client systems), and a bunch of VHDX files on an NTFS volume, copied there using VSS so you don’t have to bring your VMs down.

A big disclaimer is prominently offered — this is not production quality code. My discipline with the responsibility of memory management and other lower-level concepts is “emerging”, to use a euphemism.

Still, I thoroughly enjoyed this — it was challenging for where I currently am, but achievable. You can’t improve at something if you don’t let yourself produce output that wouldn’t perhaps yet meet your highest quality standards. (And goodness knows there is plenty of production code out there that never met those standards before it was relied upon by the world.) The perfect shall not be allowed to be the enemy of personal growth and development.

So, here is ShadowDuplicator, your very untested, extremely rudimentary VSS snapshot based backup client. Even if it’s just a workaround for a lack of vssadmin create shadow on client operating systems, it’s something. 😉

I’m delighted to announce DfontSplitter 0.4 for Windows. After a nine-year hiatus without software updates, this release has big under-the-bonnet changes!

The application is now built with .NET 4.7.2 and runs on Windows 7 – Windows 10. If you still need support going back as far as Windows 98(!), you can still use the old version.

A new, improved, fondu (which does the bulk of the work) is bundled as a DLL that is Windows-native and no longer requires the Cygwin library. It also includes a number of memory safety improvements.

To fix the long-standing issue where extracted TTFs didn’t quite play nicely with Windows, DfontSplitter 0.4 for Windows embeds functionality from FontForge to do some final conversion work to make your fonts work perfectly with Windows.

Source is available on GitHub (DfontSplitter, fondu-win-dll)

The “T with chisel” DfontSplitter icon is licensed under the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. The icon includes a modified version of “Chisel wood 24mm” by Isabelle Grosjean, which is also licensed as such.

I recently opened an old version of my DfontSplitter C# project in Visual Studio 2019. I expected a process would need to be gone through to upgrade the project from its source version (VS2008 I believe) before it could be opened in Visual Studio 2019. Such a process did happen and it completed without errors.

A little later, I noticed something odd. A folder had appeared at the root of my C: drive with the name xyDQ7sfb+k4aZLT8oSso4yqXRQ=.

Inside, a sqlite3 folder and the usual artifacts that you’d associate with a SQLite database.

At this point of my initial investigation, I hadn’t yet connected my opening of this Visual Studio project with this folder. It looked highly suspicious — a name that apparently was encoded and random.

Investigating the database

So, I fired up DB Browser for SQLite and pointed it at the database.

It contained tables including names like SolutionData2, ProjectData2, StringInfo2. Some investigation of the contents of these led me to this recent VS2008 project I’d opened.

I still felt it looked malicious though — was this something injecting itself into Visual Studio projects I was building?

Tracking down the source(?)

Some web searching later, I’d discovered a part of Roslyn, the C# compiler, which had references to these _SpellChecker_ entries I’d seen in SolutionData2 and the suggestion that this is dumped to persistent storage.

Catching it in the act

So, I deleted the folder from the C drive and replicated the process. I opened the same project again, but this time running Sysinternals Process Monitor in the background, filtering Path for sqlite or this folder name.

We can see calls to CreateFile (which creates a handle to a file — this is usually ‘opening’ the file rather than ‘creating’ it) with the result PATH NOT FOUND. Then, another call to CreateFile (highlighted) that actually creates the directory.

From there, it’s clear that this process, ServiceHub.RoslynCodeAnalysisService32.exe, is responsible for this database and its timing correlated exactly with me upgrading this project from VS2008 to VS2019.

But… what is this? Why?

I posted some Visual Studio Feedback about my observations, although this issue was closed as applying to sqlite, not Visual Studio. I understand that this isn’t high priority, but I disagree with the assessment that the use of SQLite in this case makes it SQLite’s fault!

So I remain curious. Do some code analysis on the solution, sure. But why store results in the root of the C drive? How is the encoded folder’s name decided? Why does this whole thing feel suspicious rather than behaviour I’d expect from an IDE?

And… it’s gone

As of Visual Studio 16.5.3, this appears to be fixed — the same filesystem behaviour by RoslynCodeAnalysisService32 is now prefixed with %LOCALAPPDATA%\Microsoft\VisualStudio\Roslyn\Cache\RemoteWorkspace. I guess it was indeed a bug — an oversight in actually setting the full path to this database properly.

In this post, I said

Maybe next the whole program should do the workstation locking, warning message and idle detection in one program.

Well, here it is — Idle Lock Lite.

It’s a lightweight application designed to run in the background, detecting if the computer is idle (based on keyboard or mouse movement) and presenting a timed warning that the computer will be locked. It then locks the computer if it is still idle once the grace period expires.

It is functional, but its real world usability isn’t there yet — it’s not going to be usable in practice if it isn’t aware of things like playing back video, presentations and other states that are not idle, but don’t involve user input for longer periods of time. It isn’t aware of those things at the time of writing!

However, it has been an enjoyable and informative further exploration of working with the Win32 API directly.

I found myself drawn to working in C/Win32 for this application and its predecessor — the result here is less than 50 KiB in size and uses about 1.4 MiB of RAM (Private Bytes). Any similar application you would create in .NET, for example, would be an order of magnitude more demanding in terms of resources, just because of the nature of the framework.

Here’s an explanation of how it works:

• It accepts two command line arguments — the number of seconds before displaying the warning window and the number of grace period seconds while that window is open.
• It measures the number of GetTickCount64 ticks taken for a SetTimer with 1000 ms to be fired. This is used to calculate idle periods and grace periods in terms of ticks.
• It sets up two SetWindowsHookEx hooks — one for keyboard activity and one for the mouse. These yield execution some of the time to avoid performance issues from the frequency with which they are called. These update our last interaction tick number to indicate the computer is in use.
• Every ten seconds, a SetTimer-based timer calls a function which evaluates if we’ve reached the idle condition. If so, the warning dialogue is displayed.
• The warning dialogue counts down the grace period with the progress bar and another SetTimer callback. Upon expiry, LockWorkStation is called to lock the computer.
• The WM_WTSSESSION_CHANGE window event is listened for — we take notice of WTS_SESSION_LOCK and UNLOCK events to ensure that we don’t try to detect idle conditions if the screen is already locked, and that when unlocked, any previous idle period is reset.

Learning about how Windows operates at a lower-level — windows, window messages and some of the more primitive operations has been illuminating! This is the perfect bi(y)te-sized project to help me move forward with learning about this!