Skip to content

Blog

Adventures in SELinux

Security Enhanced Linux (SELinux) is a pretty powerful technology that adds another layer of access control to a Linux system. It helps significantly limit the ability for an attacker who has partly compromised a system to use their access to jump deeper into the system.

It has been standard in Red Hat Enterprise Linux and its derivatives for quite some time, and is often the cause of many a headache when something doesn’t work because it is being (apparently) silently blocked by SELinux’s security enhancements!

Its potential to cause breakage, especially when third-party bits and pieces are brought into the mix, means the advice from well-meaning individuals is often a cry of “just turn off SELinux!”, rendering a system without that extra layer of protection.

I will not pretend that my recent dealings with SELinux in CentOS 7 have been free of frustration, but a few simple tools have made it a surprisingly simple affair to get something up and running again if a particular behaviour (always of something a bit third-party in my experience!) is being erroneously blocked.

I think a big part of what makes SELinux get switched off in frustration is the perception that it is breaking things silently, and the psychological impact of its verbose ‘scariness’ when you do find those logs!

audit2why

As long as you remember that SELinux can be the cause of potential unexplained weirdness, your first port of call can be audit2why:

audit2why -a

What is particularly nice about this tool is how quickly you get (semi-)human readable output, detailing which rules an application is breaking. If you do hit one of these ‘weird problems’, a quick trip to this tool usually makes it clear that SELinux is the cause of the failure!

audit2allow

I was surprised by how relatively quick it was to identify an issue with audit2why and make a custom module with audit2allow to get an application working again.

There are a good set of instructions in the Red Hat manual.

It sounds like a big deal, but the tools have made it almost completely automated — it really isn’t necessary to have a deep understanding of SELinux’s internal workings.

setsebool

Finally, there are some flags that are disabled by default for SELinux protected applications that you might need. Again, audit2why will often make it clear what you need to toggle using this tool!

For example, a web server process that does legitimately send out emails might need the appropriate flag switched on. Without it, the web server doesn’t get the right to talk to sendmail.

Give SELinux Another Chance!

I suppose my point in this rambling is this: give SELinux a chance if you have given up on it in the past. If you have the time to set up your system properly (and you should!), taking a little extra to figure out how to grant only the permissions really needed does make a material difference to your security should one application be compromised. An attacker being able to get their foot in the door needs to be assumed to be possible, so making their life a lot harder at that point is worth making your life slightly harder on the odd occasion.

With a little patience and the use of the tools I have talked about, I think it is a lot easier to work with than it might seem at first glance, or when it first arrived in RHEL many moons ago!

Piwik for Web Analytics

Google Analytics is almost ubiquitous as the solution for collecting useful information about how your website is being used by visitors. It is a good product, and has evolved over the years to be very flexible indeed.

But since it first launched, my opinions of Google have certainly changed, as have many others. Without wishing to get into a debate on the subject, there definitely is a market for a competitor to this very useful tool that might free us of that reliance on Google infrastructure, and be more respecting of our visitors, by means of initatives like Do Not Track.

Piwik screenshot

Piwik’s desktop and mobile views

I recently deployed Piwik, an open source PHP-based application intended to replace Google Analytics. A full disclosure — it will not be as full-featured as Google Analytics for those people using the full power of that solution, but it puts the power and control back in your hands. Moreover, it uses a very similar-looking (perhaps even largely compatible) JavaScript API, meaning I had to do little work to figure out how to track the events that I wanted.

With built-in support for avoiding the use of cookies altogether, you can sidestep the well-meaning, but ridiculously ill-conceived EU Cookie law and its onerous “we use cookies!” notifications entirely, while still delivering enough tracking capability for many simpler analytics applications where detailed insights into repeat visits aren’t so important.

I haven’t made the time to replace Google Analytics on this site with it yet, but that is on my list of things to do! Right now, I have some custom code server-side that detects your Do Not Track status and suppresses the Google Analytics JavaScript entirely, but Piwik would do away with that need for complexity.

It might not do enough for your application — but as a way to put your money where your mouth is and genuinely support the user’s right to give and withdraw consent for tracking, it is most definitely worth a look.

Serving Pi

I recently completed a physical migration of my server, the one that hosts this very page! It all went successfully, and without any noticeable downtime for this site, which I am pleased to be able to do.

There was, however, a period of time during which this server needed to be physically switched off and moved to the new location. To enable zero downtime, something would need to be able to host the server during that period.

Enter my Raspberry Pi!

Raspberry Pi in box

This amazing little thing is capable of running Raspbian, a modified version of Debian, which means I get access to the rich library of Debian packages that are available. I have a private Git repository containing a modular set of Puppet manifests. These describe the exact configuration of this server, so by applying the Puppet manifests, I can spin up a new instance of this particular server’s configuration on a whim.

So, I dusted off an SD card that was lying around, dropped Raspbian on it, and installed Puppet and Git and applied the manifests.

If I’m honest, there were a few components that weren’t quite so happy to run, despite packages being available. Varnish didn’t seem to like my VCL file, so I had to run the site here directly with Nginx pointing to PHP-FPM instead.

To cut a long story short, it worked! I was successfully serving up this site, from the Pi, using (almost) my existing configuration. Performance was not stellar, even compared to the modest hardware that normally serves this page, with page load times about 10 times slower than uncached page loads normally would be. The main blog page did take 1.5 seconds to render! For the short time I needed it though, I was very happy to have a very inexpensive and easy solution.

The Changing Face of Vulnerability News

Heartbleed logo  Shellshock Logo

The recent news about the bash vulnerability being called “ShellShock”, and the degree to which it is getting mainstream press has got me thinking about how software vulnerabilities are now being reported in the mainstream media.

Apparently, no vulnerability these days is complete without a catchy name and logo — see Heartbleed and Shellshock! Joking aside, though, the very fact that these vulnerabilities are making non-tech news headlines puts pressure on everyone running potentially vulnerable systems to do their duty — usually as straightforward as running a pre-packaged security update.

The Heartbleed and Shellshock stories are taking the place of what we used to see reserved for particularly influental computer worms, like Sasser and Mydoom. It’s most definitely positive that some vulnerabilities are getting attention — unfortunately it is still the case that for some companies and system administrators, only outside pressure will convince them to promptly, diligently and consistently apply security updates.

What I’d like to see, is some way for people interested in improving computer security, the “good guys” for lack of a better term, to leverage this media interest to send a message to system administrators that it’s always necessary to apply software updates promptly, even when they don’t get on the TV news!

The Curse of The Black Box

The other key issue that Shellshock highlights, as did Heartbleed, is the issue of embedded ‘black box’ systems that might be vulnerable. This kind of system is everywhere — and because in many cases they are ‘set it and forget it’ machines, they represent a particular risk. It’s often very difficult to convince vendors of these systems of the importance of pushing upstream software updates down to end users, particularly when there is a lack of understanding and a lack of financial incentive.

Something big and mainstream, like Shellshock and Heartbleed, might convince system administrators to badger vendors to release patches for this kind of product, but we need to extend this further, and make it a social (or even a legal) expectation on vendors to supply security updates for any product they ship, for a reasonable lifetime period for that product.

The security landscape is too complex, and everything too interconnected, for anyone to have the opinion that “I don’t need to patch that, because there’s nothing important on it”.

Leaving Yourself in the Loop

I want to part with a few bullet points, with some actions I try to take to stay up-to-date. Automatic updates are increasingly common, but not universal, and these simple things can help you not miss a known vulnerability.

  • Document and understand the whole software footprint of the systems for which you are responsible. (This means embedded systems, software libraries, and more!)
  • Subscribe to announce mailing lists, follow Short-Form “Bird” Social Media Site Before It Went Terrible accounts of the software projects and systems you use. (It pays to be in the know about available updates, and not hear about them after it is too late!)
  • Look for useful vulnerability resources for particular projects you use. (For example, for WordPress, the recently launched WPScan Vulnerability Database.)

Adventures with WindowMaker and Debian

Back in my earlier Linux days, I would experiment and fiddle a lot with different setups for desktop environments and appearance, customising my Linux system to my heart’s content! An example: I loved the 3D desktop effects of Xgl/Compiz back in 2006.

Time moved on, and I ended up settling with the defaults that distributions provided. I liked Ubuntu’s direction with Unity, upon its release in 2011.

I have fallen out with Ubuntu and Unity more recently, however. The troubling privacy issues with the Amazon ‘lens’ and other changes to their corporate behaviour scared me off.

So, I moved over to Debian for my personal server and my Linux desktop systems, and I have been very happy with it. At the same time, though, I wanted to get back to my previous spirit of playing around with different bits of software instead of just going with the defaults and surrendering to a full-size desktop environment. Frankly, the way I use Linux means I don’t find an overwhelming need for a wide variety of graphical applications.

With that in mind, I have set up a very unusual, and minimalist, desktop experience, which I thought I would document a little here for those that might be interested.

WindowMaker screenshot, showing Iceweasel, Terminal and others

» Read the rest of this post…

Upgrading to MariaDB 5.5 on CentOS 6

Installing PHP 5.5 on CentOS 6 using IUS Repositories

I have been inspired once again to fire up my screencasting rig, to show you how to install PHP 5.5 on CentOS 6 using Rackspace’s IUS Community Repositories.

More and more web applications now are likely to require versions of PHP beyond 5.3. CentOS 6 users are stuck with 5.3, with backported security updates, unless they diverge from standard repositories or compile PHP themselves! Until CentOS 7 is with us, those of us trying to run a rock-solid web server on CentOS will be left out in the cold running recent web applications like Moodle 2.7 which require a newer PHP.

In this video, I show you how to use the IUS repositories to get PHP 5.5 running. These repositories, with their Rackspace backing, seem likely to be nice and stable going forward.

As always, I’d love any feedback you might have.

Creating a Custom Child Theme in Moodle 2.6

I’m spending a fair amount of my time now working on and supporting a medium-sized Moodle installation. I will not sugar coat it: Moodle is far from my favourite piece of web software — its considerable UI complexity being my chief complaint — but it does do a reasonable job and it has a rich enough feature set to make it quite an asset in the education world.

This complexity to Moodle sometimes doesn’t exactly make it easy to do the right thing as a developer, and working with themes could be somewhere where developers diverge from best practices. The temptation to clone your favourite theme just to make a few tweaks here and there leaves people unable to track changes to the base theme, and keep their site up-to-date.

So, I have put together a video showing how you can create a custom theme (a child theme in WordPress parlance) that inherits mostly from your base theme, but allows you to override CSS and even bits of the HTML structure of Moodle’s generated pages.

I think it’s actually an easier process than people think!

As always, I welcome feedback, and if you found this particularly helpful, I’m always happy to have a few pennies drop into the PayPal account!

Find this tutorial useful?





How to install Cacti on CentOS 6

It has been far too long since a video tutorial made its debut here, so I would like to introduce a new tutorial!

Cacti is a great graphing and monitoring tool, but I have struggled in the past with getting it installed, and getting it to do what I want. It can be a little bit complex and fiddly, but recently I have had more success and am putting it to good use measuring and graphing more things.

In this tutorial, I will walk you through installing Cacti on a basic CentOS 6 system with Apache, PHP and MySQL already installed. By the end of the video, it is collecting information for the default graphs in the default installation.

I hope to extend this video series soon with some details about the additional graphs I have recently succeeded at getting installed.

As always, your comments and feedback are appreciated!

Cleaning up the IP.Board url4short mess

XDebug to the rescue…

The condensed, I-just-want-to-fix-my-site version:

On your server, try:

grep ‐ri \$mds /wherever/your/website/folder/is

to locate the injected code, and while file it resides in. You can then go into that file and remove it.

Also try re-caching all the skins and languages in the Admin Control Panel. Make sure all IP.Board updates and patches are applied to prevent the compromise happening again.

Reset your passwords and keys. Take measures to detect and continue detecting other infiltrations.

My friend Niall Brady dropped me an email, saying that some of the users of his Windows-Noob forums were reporting getting redirected to a spammy-looking site (url4short dot info) when clicking on search results to the site.

The forums run the Invision Power Board (IP.Board) software. There had been some reports of vBulletin boards being hit with this kind of spammy redirect, but fewer suggestions that this was an IPB problem. There had been a patch for a critical IPB issue released in December, but that had, obviously, been applied to the site as part of normal good practice.

Nevertheless, I was concerned. Clicking on a search engine result should definitely not be redirect somewhere other than the result page!

Without evidence that the issue was not limited to one machine, or one connection, however, it could not be ruled out that it was just malware on that visitor’s machine.

» Read the rest of this post…