Category Archives: Uncategorized

Sandy Bridge Woes – LGA1155 P67/H67 and VT-d (I/O Virtualization)

I recently upgraded my home server from an old Athlon X2 to the latest and finest from intel, namely the new Sandy bridge processors. These new processors come with a relatively new technology called VT-d or (Intel Virtualisation Technology with Directed I/O). It’s also called an IOMMU. This feature like the more common VT-x provides hardware support to improve virtualisation. VT-d does to PCI devices what VT-x does for the CPU, by virtualising and allowing remapping of the DMA. Basically, what it means is that you can now freely assign different PCI connected devices (like graphics card, USB host controllers) to your Virtual Machines. A use case would be to give a VM a real graphics card so it doesn’t have to rely on the slow virtualised graphics adapter.

It’s been a bit of a mystery whether VT-d is a CPU technology or a chipset/motherboard technology, but it’s clear that, like VT-x, you need both to know about this technology to work. Since the advent of i7s, various people have asking the support for this technology only to find that the main blocker is getting the right motherboard. Some lucky folks have gotten it as far back as dying days of the Core 2 era. Most i5s and i7s CPUs claim support for VT-d, however motherboards of P55 and H55 that support this was few and far between. The only sure way to find out if your motherboard supports it is if you find the option “Enable I/O Virtualisation” in the Advanced Features section of your BIOS. The situation markedly improved when P57 and H57 came about with MSI and Asus boards displaying the option. MSI’s H57M-ED65 is one such board. If you look at the downloadable manual, you’ll find the reference. Gigabyte is noticeably quiet on the matter. Armed with this knowledge I went out assuming that newer boards will follow the trend. How wrong was I.

There are special Sandy Bridge CPUs in the form of 250K and 260K. The K CPUs have a higher clock for the graphics, but sacrifices VT-d technology. Fortunately this is widely understood and reported, unlike last generation when confusion between VT, VT-x and VT-d reigned. What they failed to clarify was that, VT-d is unavailable in pretty much all P67/H67 chipsets. This means nobody can actually use VT-d anyway regardless of whether the CPU supports it or not. In fact, I don’t even know why Intel bother listing VT-d as a feature when support is so poor. Currently, the only sandy bridge (LGA1155) motherboard in existence, that supports VT-d is Intel’s DP67BG. I of course didn’t buy that board and as I’m building a server, I required integrated graphics and P67 doesn’t offer that. Only H67 does. As far as I can tell no H67 motherboard currently in existence has VT-d.

Anyway, this is my rant of the night.

Dictionary Model Binder in ASP.NET MVC2 and MVC3

In a decidedly typical turn of events, Microsoft changed the API of BindModel in ASP.NET MVC 2 such that it breaks DefaultDictionaryBinder. No longer can you enumerate through the ValueProvider, instead you can only Get a value which you know the name of. I’ve updated the code to work with MVC2 and also tested it with the new MVC 3 RC.

The code is compatible with ASP.NET MVC 1, 2 and 3. To use it for ASP.NET MVC 1, just set the conditional compiler directive ASPNETMVC1 and it will enable the MVC 1 code, otherwise it will work with MVC version 2 and 3.

The code is now up at github: DefaultDictionaryBinder.cs.

There’s also an example MVC3 project showing the basic functionality of the Dictionary Binder: link

Why is syslog-ng taking up 100% of CPU inside a lxc container

While experimenting with LXC, the linux virtual container, which by the way is shaping up to be a viable replacement for openvz, I ran into an annoying issue of syslog-ng taking up 100% of CPU time inside the container. Stumped, I tried to add the -d flag to the syslog command line, but it did not yield any clues.

Armed with strace, and attaching to the rouge process, the following spat out of the console again and again.

gettimeofday({1287484365, 501293}, NULL) = 0
lseek(8, 0, SEEK_END)                   = -1 ESPIPE (Illegal seek)
write(8, "Oct 19 19:39:57 login[439"..., 105) = -1 EAGAIN (Resource temporarily unavailable)

The key lines were lseek and write, both trying to write to file descriptor 8. To find out what fd 8 was, all I had to do was ls -al /proc/7411/fd/8 – The culprit was /dev/tty12. Now having looked into syslog-ng.conf, I was reminded of the fact that By default messages are logged to tty12.... So it seems, tty12 is somehow denying access to syslog. Being in LXC, I decided to check out tty12 by doing lxc-console -n container -t 12. To my surprise, syslog-ng was instantly unclogged as log messages were released into console. It looked as if the tty12 buffer was clogged up.

Regardless of the reason, the easy fix is to stop syslog-ng logging to tty12 as I’m never going look at that far away console. Commenting the console_all lines, all was fixed. This would probably never have happened if I had used metalog :/

Qemu/KVM sometimes not registering Mouse Clicks when used over VNC

After setting up Qemu/KVM and VNC and fixing cursor positioning issues (with the -usbtablet option), I had an annoying issue of the VNC viewer (TightVNC in this case) sometimes missing mouse clicks. You would quickly click on a button and icon and nothing would happen. If you hold it for long enough, it will eventually register. I don’t want to be holding my button for a second to make sure every click regsiters though.

After fiddling around with the options, I finally found the culprit. The option inside the VNC viewer “Emulate 3-buttons (with 2-button click)” seems to be the cause. Turning it off seems to make my mouse clicks reliable. No idea why though.

Getting the version number of your own Chrome Extension

UPDATE As pointed out by commenter Andreas, you can now use a simpler way:
chrome.runtime.getManifest().version
The code below no longer is necessary but kept as reference.

Following on from yesterday’s post about getting the version number of your own firefox extension, what if you were now developing a Google Chrome extension and want the same thing? Google Chrome’s extension API is much more limited that Firefox’s. There’s no explicit extension-metadata-getting API that I know of. However, we do know that the version information is tucked away in manifest.json. With this knowledge and coupled with a few friendly APIs (XMLHttpRequest & JSON.parse) we can now have the equivalent function for chrome:

[js]
function getVersion(callback) {
var xmlhttp = new XMLHttpRequest();
xmlhttp.open(‘GET’, ‘manifest.json’);
xmlhttp.onload = function (e) {
var manifest = JSON.parse(xmlhttp.responseText);
callback(manifest.version);
}
xmlhttp.send(null);
}

// to use
var version;
getVersion(function (ver) { version = ver; });
// version is populated after an indeterminate amount of time
[/js]

As XMLHttpRequest is asynchronous, our method needs a callback to receive the version information. You can also get whatever other information you want in your mainfest.json. So there you go.

Verifying Anti-Forgery Token for Form POST only

As many of you know, the ASP.NET MVC Anti-Forgery token helps thwart Cross Site Request Forgery attacks. Any site that uses authenticated sessions (99% of web apps) should use similar mechanisms so these attacks cannot occur.

Very often, I would write GET and POST actions in the same method. This allows fall through to that same code we used for GET request if POST validation fails, ensuring consistency.
[csharp]
public ActionResult EditPerson(Person person)
{
if (Request.HttpMethod == "POST" && ModelState.IsValid)
{
// do edit person…

return RedirectToAction("Index");
}

// do get person

return View(person);
}
[/csharp]
If I use that sort of paradigm, then the [VerifyAntiForgeryToken] attribute would block both GET and POST requests when the token is not supplied. I want the token to be only verified when I POST. Since ASP.NET MVC is extensible, the normal way to go about that would be modify the behaviour of [VerifyAntiForgeryToken] by subclassing. Unfortuantely, VerifyAntiForgeryTokenAttribute is sealed which means it can’t be inherited from. Luckily, borrowing from the same trick that Http*Base classes use to combat sealed Http* classes, we can just create a new attribute that wraps the old attribute, implementing the same members and proxying the calls back to the wrapped base class. That’s exactly what I did and it works quite well. The result is [ValidateAntiForgeryTokenOnPost] which will only verify the form anti-forgery token on a POST request:
[csharp]
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, AllowMultiple = false, Inherited = true)]
public class ValidateAntiForgeryTokenOnPostAttribute : FilterAttribute, IAuthorizationFilter
{
private ValidateAntiForgeryTokenAttribute _wrapped;
public ValidateAntiForgeryTokenOnPostAttribute()
{
_wrapped = new ValidateAntiForgeryTokenAttribute();
}

public string Salt { get { return _wrapped.Salt; } set { _wrapped.Salt = value; } }

public void OnAuthorization(AuthorizationContext filterContext)
{
if (filterContext.HttpContext.Request.HttpMethod == "POST")
{
_wrapped.OnAuthorization(filterContext);
}
}
}
[/csharp]
This will teach those sealed classes!

iptsafe – iptables with dead man’s switch

When dealing with iptables remotely, you can easily set a firewall rule which would lock yourself out of machine. After that, the only way to unlock yourself is to physically go the machine and unset the firewall rules it through the terminal. If this was a VPS or dedicated server, chances are you can’t physically access the machine and have to contact the service provider to reset the firewall rules.

This is an instance where a dead man’s switch would help. The theory goes that if an operator is detected incapacitated, then an certain action will occur. In our case, the action is to undo our firewall changes. How does it know we are incapacitated? Well if we don’t report back in a certain amount of time, then we’re probably dead. A long time application of this is actually found when you try to change your monitor’s resolution, it ask you if you want to keep it. If it gets no response, it’ll revert back automatically.

Do you want to keep these settings

The iptsafe script works on the same principle. It is a wrapper around the iptables command. It takes the same command parameters as iptables, with the exception that if you only specify one parameter, then it’ll assume it is an iptables-save’d file and use it on iptables-restore. Once iptsafe is run, it’ll first use iptables-save to store a copy of the current iptables state, then apply the changes you requested. After that, it prompts you to keep the changes, and if you don’t respond within 15 seconds, it’ll revert back to the original state.

Here’s iptsafe

Sample usage:
# iptsafe -A INPUT -i eth0 -p tcp -s 192.168.0.1 -j ACCEPT
or
# iptsafe my-saved-iptables

An intuitive Dictionary Model Binder for ASP.NET MVC

The other day I was working on an ASP.NET MVC website and came across a need to post an array from the browser into the web app. The framework comes with something called a Model Binder that automagically converts submitted form data into action parameters of the controller. For example, if we have form submitted data such as
person.FirstName=John&person.LastName=smith
for a theoretical model class ‘Person’, and
[csharp]public ActionResult SavePerson(Person person)[/csharp]
as the action method signature, SavePerson will be executed with the parameter equivalent to
[csharp]new Person() { FirstName = "John", LastName = "Smith" }[/csharp]

The default model binder is pretty powerful, using reflection to dig out and assign all the fields. It also supports arrays and dictionaries, but with big limitations. The array must start at 0 and be unbroken. That is understandable for arrays, but what if you had a dictionary? Surely it can start at any position? Not so. The dictionary has even more obscure requirements, with the need to specify explicit .Key and .Value parameters in your form submission. For example:
dict[0].Key=mykey&dict[0].Value=myvalue
This represented extra work to generate the form on the client side. I just want to input something more intuitive like:
dict[mykey]=myvalue
The ASP.NET MVC framework is highly extensible. It allows you to define your own custom model binder so that’s exactly what I did. Inheriting off DefaultModelBinder, I created DefaultDictionaryBinder that overrode the BindModel method and intercepts when a IDictionary<,> class is being bound.

The code is now up at github: DefaultDictionaryBinder.cs. Note that if you are using this on ASP.NET MVC 1, please define the macro ASPNETMVC1. If you are using it with MVC 2 or MVC 3, it should work as is.

To use, you have to override the default model binder. In global.asax.cs in Application_Start(), add the line:
[csharp]
ModelBinders.Binders.DefaultBinder = new DefaultDictionaryBinder();
[/csharp]

The code is very flexible, only requiring the dictionary key to be of a basic type convertible from string, ie. Dictionary or Dictionary. The value can be any object that is able to be bound by the default model binder.
An example follows:
If your form input is
persons[3].FirstName=John&persons[3].LastName=Smith&persons[4].FirstName=Jane&persons[4].LastName=Doe&
and our action signature
[csharp]public ActionResult SavePersons(Dictionary<int, Person> persons)[/csharp]
the persons parameter would be
[csharp]
new Dictionary<int, Person>() {
{ 3, new Person() {"John", "Smith"} },
{ 4, new Person() {"Jane", "Doe"} },
}
[/csharp]

Simple and intuitive.

Download: DefaultDictionaryBinder.cs Simple Example Project (ASP.NET MVC 3 required)

Thinking about Geolocation APIs for Windows Mobile

Since owning a WinMo phone (or windows phone?), I’ve been thinking of developing some real applications on it. Currently, Geolocation is all the rage, allowing a application to respond different depending on your current location. This is done automatically via different mechanisms such as the most obvious and accurate one, GPS, and less reliable ones such as Cell Tower ID and nearby WIFI hotspot mac addresses.

So I set about searching for a Geo-Location API for windows mobile. Unfortunately, compared to Android and iPhone, the WinMo API is severely lacking in this regard. There seems to be some support in the next version of the .NET Compact Framework (4), but nothing in the current released APIs.

So I set about looking at third party APIs, most notably Google Gears. Google Gears is a framework that plugs into browsers, giving web pages expanded functionality such as offline storage. It also provides what appears to be, a comprehensive Location API, supporting gps, cell ID, and wifi for positioning and returning the Longitude/Latitude and street address. There are 2 gear plugins available for Windows Mobile, one for IE Mobile and one for Opera Mobile. The gears framework API seems to be available only to web browsers, thus web page developers, but not normal application developers.

So I set about seeing if I can access the google location web service directly. Helpfully, google posts the web service API specifications to build your own location API service provider. Unfortunately, it doesn’t tell you where the google service end point is. According to this page, google only allows interaction with the location service via Gears API. So it seems that even if you sniffed out the end point URL, you’ll be in violation of the google TOS to use the service.

But all is not lost – a workaround to all this, is to somehow embed Google Gears in your own application. Since google gears is open source software, BSD licensed, you can freely embed and distribute it along with your application. Searching the web, there seems to be very little information regarding embedding. Looking at the source, there are code relating to integrating with specific browsers. Ideally, we should link with a distributed google gears runtime by emulating an already defined browser interface. The NPAPI interface seems to be the most hopeful. I suspect Gears for Opera Mobile uses the NPAPI interface.

My aim is to provide a Windows Mobile .NET CF interface to the location api of google gears. If I am successful in that endeavour, there will a part two of this post. If not, I hope this post helps someone out there.

Quick tip: Why is my service binding to ipv6 localhost instead of ipv4?

So you just got a brand new VPS and installed a service, say PostgreSQL onto it. You then run your web application and found it complaining that it can’t connect to 127.0.0.1 on port 5432. Since you’re a seasoned administrator, you verify it by doing netstat -an. To your surprise, you find postgresql is binding to ::1:5432, the ipv6 localhost address. You curse the VPS provider for enabling and giving you an ipv6 network interface. You don’t even need ipv6 until 2050! But all is not lost. The reason why postgresql is binding to ::1:5432 is because it binds to localhost, which points to both ::1 (ipv6) and 127.0.0.1 (ipv4). The solution is simple. Open up /etc/hosts and find a line that says
[code]::1 localhost[/code]
and comment it out. Restart postgresql or any other service and enjoy dotted-quad goodness again.