Michael Friis' Blog

Podnanza is a simple screen-scraper/feed-generator that I built for my own amusement to podcast shows from Danish Radio’s (DR) Bonanza archive. Check out the Podnanza announcement post for details. This post describes how Podnanza was built using ASP.NET Core running on AWS Lambda. The Podnanza source code is on GitHub.

I’ll start by admitting that AWS Lambda is the wrong technical architecture for Podnanza. Nothing’s wrong with Lambda, but Podnanza is a set of very static RSS feeds: The shows are from DR’s archive and don’t change or get new episodes. A simpler Podnanza implementation would have been a static-site generator that scraped the archive and put the XML RSS feed files in AWS S3.

I opted for Lambda for the very bad reason that I wanted to learn about serverless/function-based development by implementing a “real” project, and Podnanza was the realest small-size idea on my mind at the time. At least it’ll only be me that has to deal with maintenance of the over-complicated setup.

FaaS and HTTP Apps

Working (as I do) on PaaS/FaaS/Serverless products one might encounter arguments like:

FaaS is event-based programming and HTTP requests can be thought of as events. If a PaaS platform has good autoscaling and scale-to-zero (idling) then separate FaaS-features are not needed—people should just build FaaS apps as normal HTTP services.

Or the other way around:

If we have FaaS and event-based programming, why would we also support long-running processes for serving HTTP requests? People should just build HTTP apps from FaaS features since dealing with HTTP requests is an example of handling events

In the abstract, both of these of these statements are correct but they also obscure a lot of useful nuances. For example, even the slickest HTTP app platform pushes some HTTP handling overhead onto developers. Programs that only have to accept events through an interface defined in an SDK maintained by the FaaS platform can be a lot simpler than programs dealing with HTTP, even when a HTTP endpoint is only required for ingesting events. And because event-handling is a more constrained problem than general HTTP, platform-provided tooling such as SDKs and test-mocks can be more targeted and effective.

Similarly, forcing all HTTP apps to be built by handling events coming through a FaaS platform event interface is not ideal either:

• Lots of apps have already been built using HTTP frameworks like Node.js Express, and those apps would have to be rewritten to conform to the event interface
• Many developers are very experienced and productive building HTTP apps using existing HTTP frameworks and it’s not worth it for them to ditch those frameworks for an event-based HTTP model, even if it comes with slightly reduced management overhead
• FaaS interfaces are still largely proprietary and platform-specific, causing lock-in (although middleware like the Serverless Framework can help mitigate that). HTTP apps, on the other hand, can run anywhere

ASP.NET Core on AWS Lambda

With all that out of the way, let’s look at how AWS made ASP.NET Core respond to HTTP requests on Lambda. Spoiler alert: It’s a pretty clever blend of the two dogmas outlined above.

Generally serverless “web apps” or APIs are built with Lambda by using an AWS API Gateway (optionally combined with CloudFront for CDN and S3 for static assets) that sends API Gateway Message Events to a Lambda function. The events are basically JSON-formatted HTTP requests, and the HTTP “response” emitted by the function is also JSON formatted. Building a serverless .NET web app on top of that would be pretty frustrating for anyone familiar with ASP.NET because all of the HTTP, MVC, routing and other tooling in ASP.NET would not work.

But here’s the genius: Because the ASP.NET Core framework is fairly well-factored AWS was able to build a HTTP request pipeline frontend (Amazon.Lambda.AspNetCoreServer) that marshals API Gateway Message Events and feeds them into the rest of ASP.NET Core as if they were normal HTTP requests (which, of course, they were before the AWS API Gateway messed them up and turned them into JSON). The AWS blog post has more details and also diagrams (reproduced below) showing the two execution models.

The result is that ASP.NET Core web apps can be debugged and tested locally using the “standard” IIS/Kestrel-based pipeline and then built and deployed using the Amazon.Lambda.AspNetCoreServer based pipeline for production deploys to AWS Lambda. AWS even ships Visual Studio plugins and dotnet new templates that make getting started simple.

While neat, the Lambda approach completely ignores the ideal of dev/prod parity and the execution framework during local testing (with IIS/Kestrel) is very different from the production environment. Somewhat to my surprise I encountered zero problems or abstraction-leaks with the exotic HTTP setup when building and evolving Podnanza, but I suspect that more complex apps that make fuller use of HTTP semantics might see oddities.

Summary

Podnanza has been running without a hitch on AWS Lambda for more than 6 months at the time this post was written, typically costing around $0.20/month including CloudFront and API Gateway use. I’ve pushed multiple tweaks and improvements without issue during that time, always using the dotnet lambda package. On a side-note I admire the AWS .NET team’s zeal in building the Lambda deploy flow into the dotnet tool, but I wonder if it would have made more sense to just add it to the aws CLI that developers use to complete other AWS tasks. Also note that I haven’t built any CI/CD or GitHub-based deployment flow since it’s just me working on and deploying Podnanza. Maybe improving that would be a good way to learn about GitHub Actions…

This blog post presents a Docker Language Stack for creating and running ASP.NET 5 (née vNext) apps. It’s based on my work last week to run ASP.NET 5 on Google Container Engine.

I the interim, the ASP.NET team has released their own Docker image. It’s not really up to spec for being a Docker language stack though, so I forked it, added what was missing and published it on Docker Hub.

Other people already sent PRs to add onbuild support to the ASP.NET repo, but there’s apparently some uncertainty about how ASP.NET 5 apps are going to get built, so they’re holding off on merging. I hope that eventually the work presented here will get folded into the official repo, just like it happened with the Mono stack I created a month ago. That’s the base for what’s now the official Mono Docker language stack, which, incidentally, is what the ASP.NET docker image derives from!

How to use

Using the onbuild image is pretty simple. To run HelloWeb sample, clone that repo and add this Dockerfile in the HelloWeb dir, next to the project.json:

FROM friism/aspnet:1.0.0-beta1-onbuild
EXPOSE 5004

Now build the image:

docker build -t my-app .

And finally run the app, exposing the site on port 80 on your local machine:

docker run -t -p 80:5004 my-app

Note that the -t option is currently required when invoking docker run. This is because there’s some sort of bug in Kestrel that requires the process to have a functional tty to write to – without a tty, Kestrel hangs on start.

Speed is a feature and one thing that can slow down web apps is clients waiting to download images. To speed up image downloads and conserve bandwidth, the good folks of Google have come up with a new image format called WebP (“weppy”). WebP images are around 25% smaller in size than equivalent images encoded with JPEG and PNG (WebP supports both lossy and lossless compression) with no worse perceived quality.

This blog post shows how to dynamically serve WebP-encoded images from ASP.NET to clients that support the new format.

One not-so-great way of doing this is to serve different HTML depending on whether clients supports WebP or not, as described in this blog post. As an example, clients supporting WebP would get HTML with <img src="image.webp"/> while other clients would get <img src="image.jpeg"/>. The reason this sucks is that the same HTML cannot be served to all clients, making caching harder. It will also tend to pollute your view code with concerns about what image formats are supported by browser we’re rendering for right now.

Instead, images in our solution will only ever have one url and the content-type of responses depend on the capabilities of the client sending the request: Browsers that support WebP get image/webp and the rest get image/jpeg.

I’ll first go through creating WebP-encoded images in C#, then tackle the challenge of detecting browser image support and round out the post by discussing implications for CDN use.

Serving WebP with ASP.NET MVC

For the purposes of this article, we’ll assume that we want to serve images from an URI like /images/:id where :id is some unique id of the image requested. The id can be used to fetch a canonical encoding of the image, either from a file system, a database or some other backing store. In the code I wrote to use this, the images are stored in a database. Once fetched from the backing store, the image is re-sized as desired, re-encoded and served to the client.

At this point, some readers are probably in uproar: “Doing on-the-fly image fetching and manipulation is wasteful and slow” they scream. That’s not really the case though, and even if it were, the results can be cached on first request and then served quickly.

Assume we have an Image class and method GetImage(int id) to retrieve images:

private class Image
{
	public int Id { get; set; }
	public DateTime UpdateAt { get; set; }
	public byte[] ImageBytes { get; set; }
}

We’ll now use the managed API from ImageResizer to resize the image to the desired size and then re-encode the result to WebP using Noesis.Drawing.Imaging.WebP.dll (no NuGet package, unfortunately).

public ActionResult Show(int imageId)
{
	var image = GetImage(imageId);

	var resizedImageStream = new MemoryStream();
	ImageBuilder.Current.Build(image.ImageBytes, resizedImageStream, new ResizeSettings
	{
		Width = 500,
		Height = 500,
		Mode = FitMode.Crop,
		Anchor = System.Drawing.ContentAlignment.MiddleCenter,
		Scale = ScaleMode.Both,
	});

	var resultStream = new MemoryStream();
	WebPFormat.SaveToStream(resultStream, new SD.Bitmap(resizedImageStream));
	resultStream.Seek(0, SeekOrigin.Begin);

	return new FileContentResult(resultStream.ToArray(), "image/webp");
}

System.Drawing is referenced using using SD = System.Drawing;. The controller action above is fully functional and can serve up sparkling new WebP-formatted images.

Browser support

Next up is figuring out whether the browser requesting an image actually supports WebP, and if it doesn’t, respond with JPEG. Luckily, this doesn’t involve going back to the bad old days of user-agent sniffing. Modern browsers that support WebP (such as Chrome and Opera) send image/webp in the accept header to indicate support. Ironically given that Google came up with WebP, the Chrome developers took a lot of convincing to set that header in requests, fearing request size bloat. Even now, Chrome only advertises webp support for requests that it thinks is for images. In fact, this is another reason the “different-HTML” approach mentioned in the intro won’t work: Chrome doesn’t advertise WebP support for requests for HTML.

To determine what content encoding to use, we inspect Request.AcceptTypes. The resizing code is unchanged, while the response is generated like this:

	var resultStream = new MemoryStream();
	var webPSupported = Request.AcceptTypes.Contains("image/webp");
	if (webPSupported)
	{
		WebPFormat.SaveToStream(resultStream, new SD.Bitmap(resizedImageStream));
	}
	else
	{
		new SD.Bitmap(resizedImageStream).Save(resultStream, ImageFormat.Jpeg);
	}

	resultStream.Seek(0, SeekOrigin.Begin);
	return new FileContentResult(resultStream.ToArray(), webPSupported ? "image/webp" : "image/jpeg");

That’s it! We now have a functional controller action that responds correctly depending on request accept headers. You gotta love HTTP. You can read more about content negotiation and WebP on lya Grigorik’s blog.

Client Caching and CDNs

Since it does take a little while to perform the resizing and encoding, I recommend storing the output of the transformation in HttpRuntime.Cache and fetching from there in subsequent requests. The details are trivial and omitted from this post.

There is also a bunch of ASP.NET cache configuration we should do to let clients cache images locally:

	Response.Cache.SetExpires(DateTime.Now.AddDays(365));
	Response.Cache.SetCacheability(HttpCacheability.Public);
	Response.Cache.SetMaxAge(TimeSpan.FromDays(365));
	Response.Cache.SetSlidingExpiration(true);
	Response.Cache.SetOmitVaryStar(true);
	Response.Headers.Set("Vary",
		string.Join(",", new string[] { "Accept", "Accept-Encoding" } ));
	Response.Cache.SetLastModified(image.UpdatedAt.ToLocalTime());

Notice that we set the Vary header value to include “Accept” (as well as “Accept-Encoding”). This tells CDNs and other intermediary caching proxies that if they try to cache this response, they must vary the cached value based on value of the “Accept” header of the request. This works for “Accept-Encoding”, so that different values can cached based on whether the response is compressed with gzip, deflate or not at all, and all major CDNs support it. Unfortunately, the mainstream CDNs I experimented with (CloudFront and Azure CDN) don’t support other Vary values than “Accept-Encoding”. This is really frustrating, but also somewhat understandable from the standpoint of the CDN folks: If all Vary values are honored, the number of artifacts they have to cache would increase at a combinatorial rate as browsers and servers make use of cleverer caching. Unless you find a CDN that specifically support non-Accept-Encoding Vary values, don’t use a CDN when doing this kind of content negotiation.

That’s it! I hope this post will help you build ASP.NET web apps that serve up WebP images really quickly.