Skip to content

Blog

Emerging from the Shadows

I have been getting back into the swing of building things just for fun and for exploration. I think a pressure can emerge that the things we create have to mean something, have to hit some mark of quality to be worthy of pursuing at all. I think that is a mistake, especially in that it discourages you from being open to areas where you don’t already have great expertise. So, here I am, putting into practice rejecting that pressure.

Continuing my theme of trying to work with lower-level APIs and with unmanaged languages like C and C++, I have developed a very simple client for Windows’ Volume Shadow Copy service, that allows me to create a shadow copy of a volume, copy some files in a folder to a destination, and then release the shadow copy.

I can see a use for this as part of a very low-tech backup solution where a drive is (most of the time) physically disconnected, and where you want to keep the technology stack as simple as humanly possible for the greatest flexibility in disaster recovery scenarios — BitLocker for external drives (compatible even with Windows client systems), and a bunch of VHDX files on an NTFS volume, copied there using VSS so you don’t have to bring your VMs down.

A big disclaimer is prominently offered — this is not production quality code. My discipline with the responsibility of memory management and other lower-level concepts is “emerging”, to use a euphemism.

Still, I thoroughly enjoyed this — it was challenging for where I currently am, but achievable. You can’t improve at something if you don’t let yourself produce output that wouldn’t perhaps yet meet your highest quality standards. (And goodness knows there is plenty of production code out there that never met those standards before it was relied upon by the world.) The perfect shall not be allowed to be the enemy of personal growth and development.

So, here is ShadowDuplicator, your very untested, extremely rudimentary VSS snapshot based backup client. Even if it’s just a workaround for a lack of vssadmin create shadow on client operating systems, it’s something. 😉

DfontSplitter 0.4 for Windows

I’m delighted to announce DfontSplitter 0.4 for Windows. After a nine-year hiatus without software updates, this release has big under-the-bonnet changes!

The application is now built with .NET 4.7.2 and runs on Windows 7 – Windows 10. If you still need support going back as far as Windows 98(!), you can still use the old version.

A new, improved, fondu (which does the bulk of the work) is bundled as a DLL that is Windows-native and no longer requires the Cygwin library. It also includes a number of memory safety improvements.

To fix the long-standing issue where extracted TTFs didn’t quite play nicely with Windows, DfontSplitter 0.4 for Windows embeds functionality from FontForge to do some final conversion work to make your fonts work perfectly with Windows.

Source is available on GitHub (DfontSplitter, fondu-win-dll)

The “T with chisel” DfontSplitter icon is licensed under the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. The icon includes a modified version of “Chisel wood 24mm” by Isabelle Grosjean, which is also licensed as such.

PFSense 2.4.5 on Hyper-V following 2020-05 Cumulative Update

Following applying the 2020-05 Cumulative Update for Windows Server 2016 (1607), a PFSense 2.4.5 Hyper-V virtual machine on this host OS suddenly became very unstable.

The machine appears to experience high load averages immediately following boot, with a large percentage of the vCPU usage (>99%) being shown as “system” in top.

Additionally, sometimes it is seen that the pfctl process is also using close to 100% of a CPU. The system is unresponsive, or intermittently responsive throughout this time — both in terms of the console, the web UI, and packet forwarding that the machine should be doing.

After a few minutes, Hyper-V appears to restart the guest OS. This isn’t a PFSense kernel panic and no dump is made by PFSense.

I spun up a new VM of the unstable daily snapshot of PFSense, a Generation 2 UEFI-based VM. This completely eliminated these problems for us in this new VM — it works fine, routes just fine for what we’re doing and is responsive.

I don’t have a solution — I just want to throw this issue out there in case others have experienced it, in case we might work together to figure out what’s going on here.

RoslynCodeAnalysisService creating SQLite Databases on C:

I recently opened an old version of my DfontSplitter C# project in Visual Studio 2019. I expected a process would need to be gone through to upgrade the project from its source version (VS2008 I believe) before it could be opened in Visual Studio 2019. Such a process did happen and it completed without errors.

A little later, I noticed something odd. A folder had appeared at the root of my C: drive with the name xyDQ7sfb+k4aZLT8oSso4yqXRQ=.

The unusual looking folder name contained a SQLite database.

Inside, a sqlite3 folder and the usual artifacts that you’d associate with a SQLite database.

At this point of my initial investigation, I hadn’t yet connected my opening of this Visual Studio project with this folder. It looked highly suspicious — a name that apparently was encoded and random.

Investigating the database

So, I fired up DB Browser for SQLite and pointed it at the database.

It contained tables including names like SolutionData2, ProjectData2, StringInfo2. Some investigation of the contents of these led me to this recent VS2008 project I’d opened.

I still felt it looked malicious though — was this something injecting itself into Visual Studio projects I was building?

Tracking down the source(?)

Some web searching later, I’d discovered a part of Roslyn, the C# compiler, which had references to these _SpellChecker_ entries I’d seen in SolutionData2 and the suggestion that this is dumped to persistent storage.

Catching it in the act

So, I deleted the folder from the C drive and replicated the process. I opened the same project again, but this time running Sysinternals Process Monitor in the background, filtering Path for sqlite or this folder name.

We can see calls to CreateFile (which creates a handle to a file — this is usually ‘opening’ the file rather than ‘creating’ it) with the result PATH NOT FOUND. Then, another call to CreateFile (highlighted) that actually creates the directory.

From there, it’s clear that this process, ServiceHub.RoslynCodeAnalysisService32.exe, is responsible for this database and its timing correlated exactly with me upgrading this project from VS2008 to VS2019.

But… what is this? Why?

I posted some Visual Studio Feedback about my observations, although this issue was closed as applying to sqlite, not Visual Studio. I understand that this isn’t high priority, but I disagree with the assessment that the use of SQLite in this case makes it SQLite’s fault!

So I remain curious. Do some code analysis on the solution, sure. But why store results in the root of the C drive? How is the encoded folder’s name decided? Why does this whole thing feel suspicious rather than behaviour I’d expect from an IDE?

And… it’s gone

As of Visual Studio 16.5.3, this appears to be fixed — the same filesystem behaviour by RoslynCodeAnalysisService32 is now prefixed with %LOCALAPPDATA%\Microsoft\VisualStudio\Roslyn\Cache\RemoteWorkspace. I guess it was indeed a bug — an oversight in actually setting the full path to this database properly.

Installing OpenWRT 19.07.2 on TP-Link WA801ND v5

I recently had some success installing OpenWRT 19.07.2 on my TP-Link WA801ND v5 wireless access point, which I was happy to do since the stock firmware was untouched since 2017.

I’ve documented the process in a video. This isn’t a tutorial with absolutely everything covered — I don’t want to encourage people to brick their devices, so you’ll need competence and proficiency with configuring your network settings in your operating system, and running a TFTP server.

It’s all, obviously, at your own risk. Stay safe and don’t brick your only access point!

Sorry for the inconsistency in audio quality.

Provisioning Raspbian with WiFi and SSH

Raspberry Pi logo

UPDATE: This does not work since Raspberry Pi OS based on bullseye. sdm might be a good alternative approach.

I’ve been playing with some Raspberry Pi Zero W machines for a few projects. They are inexpensive, with a form factor that makes for all sorts of interesting possibilities.

They are also a pain to initially provision, because you need adapter cables for mini-HDMI and micro-USB to get a monitor and keyboard connected.

With some help from this post from Linuxconfig.org, here’s what I’ve been doing to get the Pi Zero W up and running, on the network and ready for SSH access without having to plug anything in except power and the SD card!

Mount the image

$ fdisk -l *raspbian.img 

The initial FAT /boot partition lies at 8192 sectors in. A sector is 512 bytes. So, now we’ll mount the /boot partition using the loop pseudo-device.

$ mkdir boot
$ mount -o loop,offset=$(( 512 * 8192)) *raspbian.img boot

Set up SSH

Let’s go into our new boot subfolder where the image’s first partition is now mounted.

All we need to do to enable SSH is create an empty file called ssh.

$ cd boot
$ sudo touch ssh

Configure WiFi

To configure WiFi, we’ll need to drop a wpa-supplicant.conf file in this folder. Upon first boot, Raspbian moves this into the correct config location and the Pi will be able to talk to our network out of the box.

Create a file called wpa-supplicant.conf with these contents:

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=GB

network={
	ssid="Your network name"
	psk="Your network password"
	key_mgmt=WPA-PSK
}

You’ll need to replace the placeholders with your network name and password, and possibly change country too.

Unmount the image

$ cd ..
$ umount boot

Write the image

Insert the target SD card and verify which device it is using fdisk.

Take great care to ensure that the command below is run with the correct device as the target, or you’ll overwrite a hard drive on your local system.

$ sudo fdisk -l

Disk /dev/sdb ...
Model: Card Reader

I am confident that sdb is correct, because I see from fdisk that the model is Card Reader.

$ sudo dd if=*raspbian.img of=/dev/sdb

Security

Remember that once you’ve booted, you’ve just spun up a Raspberry Pi on your network with the default username of pi and the default password of raspberry. Check your router or network logs for the assigned DHCP address of the new Pi, log in promptly over SSH and change that password!

Licensing: Restricting the ability to prevent others exercising their rights

There is great potential in using open source and free software licensing to support and encourage the ethical use of technology — and indeed to deter its use in violating human rights and provide some legal mechanisms to challenge this when it happens.

But it’s also an extremely complex and difficult issue, as there are always a great number of implications, and they’re not always easy to anticipate.

This is an area I watch with keen interest, and I was interested to read Matthew Garrett posing some interesting questions around how a clause relating to taking away another’s ability to exercise their rights under the licence might work. The idea is taking the moral goal of preventing harm being done to others by means of the software and expressing that goal in terms of prohibiting actions that would prevent others from exercising their rights under the same licence.

This work may not be used in any way that impairs any individual’s ability to exercise the permissions granted by this license, whether or not they have received a copy of the covered work

To be clear, I don’t think this is a good license – it has a bunch of unfortunate consequences like it being impossible to use covered code in self-defence if doing so would impair your attacker’s ability to use the software. I’m not advocating this as a solution to anything. But I am interested in seeing whether the perception of the argument changes when we refocus it on user freedom as opposed to an independent ethical goal.

Matthew Garrett

Why is a website connecting to “localhost”?? — Socky and SockyNotifier (macOS)

Inspired by Davy Wybiral’s demonstration (explanation) of how web pages can often enumerate services running on localhost using JavaScript, I put together Socky and SockyNotifier.

The idea is that you have Socky listen on target ports, and any connection attempts that come in to those target ports will fire a user notification at the top right of your screen (that’s the job of SockyNotifier — to show those notifications).

Note: that this is not particularly serious or practical, but I wanted a project that let me work directly with the Core Foundation APIs in C, and this seemed a good opportunity.

Idle Lock Lite (Going Lower-Level 2.0)

In this post, I said

Maybe next the whole program should do the workstation locking, warning message and idle detection in one program.

Well, here it is — Idle Lock Lite.

It’s a lightweight application designed to run in the background, detecting if the computer is idle (based on keyboard or mouse movement) and presenting a timed warning that the computer will be locked. It then locks the computer if it is still idle once the grace period expires.

Idle Lock Lite

It is functional, but its real world usability isn’t there yet — it’s not going to be usable in practice if it isn’t aware of things like playing back video, presentations and other states that are not idle, but don’t involve user input for longer periods of time. It isn’t aware of those things at the time of writing!

However, it has been an enjoyable and informative further exploration of working with the Win32 API directly.

I found myself drawn to working in C/Win32 for this application and its predecessor — the result here is less than 50 KiB in size and uses about 1.4 MiB of RAM (Private Bytes). Any similar application you would create in .NET, for example, would be an order of magnitude more demanding in terms of resources, just because of the nature of the framework.

Here’s an explanation of how it works:

  • It accepts two command line arguments — the number of seconds before displaying the warning window and the number of grace period seconds while that window is open.
  • It measures the number of GetTickCount64 ticks taken for a SetTimer with 1000 ms to be fired. This is used to calculate idle periods and grace periods in terms of ticks.
  • It sets up two SetWindowsHookEx hooks — one for keyboard activity and one for the mouse. These yield execution some of the time to avoid performance issues from the frequency with which they are called. These update our last interaction tick number to indicate the computer is in use.
  • Every ten seconds, a SetTimer-based timer calls a function which evaluates if we’ve reached the idle condition. If so, the warning dialogue is displayed.
  • The warning dialogue counts down the grace period with the progress bar and another SetTimer callback. Upon expiry, LockWorkStation is called to lock the computer.
  • The WM_WTSSESSION_CHANGE window event is listened for — we take notice of WTS_SESSION_LOCK and UNLOCK events to ensure that we don’t try to detect idle conditions if the screen is already locked, and that when unlocked, any previous idle period is reset.

Learning about how Windows operates at a lower-level — windows, window messages and some of the more primitive operations has been illuminating! This is the perfect bi(y)te-sized project to help me move forward with learning about this!

DfontSplitter 1.0 on the Mac App Store

I recently completed my rewrite of DfontSplitter for Mac, my tool for converting Mac-formatted Dfont, Font Suitcase and TTC font files into TTF files.

This new version is written in Swift, targets macOS Mojave (10.14) and later and meets all the requirements that soon will be required of Mac software (code signing, notarisation/delivery via Mac App Store).

It’s available on the Mac App Store, with source code now available on GitHub.

The “T with chisel” DfontSplitter icon is licensed under the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. The icon includes a modified version of “Chisel wood 24mm” by Isabelle Grosjean, which is also licensed as such.

The old “T with pencil” DfontSplitter icon is from the Oxygen Icon set and is licensed under the Creative Commons BY-SA 3.0 Unported licence. Please see the More Information pages for Windows and Mac for full licensing information.

Apple, the Apple logo and Mac are trademarks of Apple Inc., registered in the U.S. and other countries and regions. App Store is a service mark of Apple Inc.