This is just a little note to explain why the forum may not be accessible for a little while. It’s moving to a better server, although it’ll have the same http:// location.

You shouldn’t have to do anything other than be please be patient for the next day or so, and then things should be back to normal once all the new DNS stuff propagates.

Apparently we were put on the wrong server during the last migration, and just recently we’ve been bumping into all of its CPU usage limitations 🙂

Every once in awhile you get an idea for an optimization that is so simple and so effective that you wonder why you didn’t think of it much earlier. As a case in point, someone recently posted to the forum that their image was taking a long time to save. It was an image comprised of 20 layers at 1280×720 pixels, and 20 seconds was deemed to be too long. The guy had a brand new AMD Phenom II X4 960T 3.0GHz gizmo with 4 CPU cores, plenty of RAM, and an SSD. It was obviously bottlenecked on the CPU because Task Manager was pegged at 100% while saving.

Please wait forever

The only advice I could give for the latest public release of Paint.NET (v3.5.10) was that if you need it to save quicker, either 1) use fewer layers, or 2) get a faster CPU with more cores. This is very technical advice, and in some circles that works just fine especially if it’s practical and economical, or if you’re dependent on saving time in order to make money (saving 5 minutes every day is often worth $100 to buy more RAM). Using fewer layers isn’t a good solution though, and replacing the CPU usually isn’t very practical either especially if it’s already a brand new upgrade.

It got me thinking though, that Paint.NET can still do better. If you’re working with a lot of layers then chances are that each layer is mostly blank space. Lots of transparent pixels with an RGBA value of #00000000 and a simple integer value of 0. So why not find a way to short-circuit that? Paint.NET does not yet used a “tiled” memory management scheme for layer data, but there was still a very good, very simple, and very effective solution just begging to be implemented.

PDN images contain a header, a bunch of structural information (# of layers, etc.), and then a whole lot of raw pixel data compressed using GZIP. It isn’t one long GZIP stream, however. Each layer is broken up into 256 KB chunks which are compressed separately. This does increase the file size, but not by much (a percent or two), and it lets me compress each block in its own thread and makes saving a much faster ordeal than it otherwise would be. Save and load times are able to drop almost linearly with the number of CPU cores.

So yesterday, in the 4.0 code base, I fixed this scenario further. For regions of the image that contain “homogenous values,” which would be a 256 KB chunk of all the same color, I compress it once and then cache the result. The next time I find a region that has the same homogenous color pattern, I skip the compression altogether and emit the result of the previous time I compressed it. If a region isn’t “homogenous” then it’s discoverable pretty quick and therefore doesn’t waste much CPU time.

(These 256KB regions do not have any geometric shape associated with them, such as being a 128×128 tile. A 256KB chunk in an 800×600 image would be the first 81 rows, and then most of the 82nd row. The next chunk would start off near the end of the 82nd row, and then include almost the next 81 rows of pixel data.)

The result? About 50% faster save times! The first time you save the image will take the longest. After that the cache will be “warmed up” and saving is even faster, resulting in up to about a 90% reduction in save time.

This is a great optimization: there’s no change to the PDN format, so images saved in 4.0 are still completely compatible with earlier versions or with other applications that have done the grunt work to support PDN (I think GIMP does nowadays). It’s just faster.

Anyway, it’s been awhile since I posted. Fear not, work is still progressing on 4.0. Silence is just an indicator that I’ve been really busy, not abandonment. Ed Harvey has contributed a few really cool features that have been popular requests, such as the ability to use the Color Picker tool to sample all layers, and to sample the average from a 3×3, 5×5, 11×11, 33×33, or 55×55 region (apparently that’s what Photoshop does). He also added a few enhancements to the color wheel so you can use Ctrl, Alt, or Shift to constrain the hue, saturation, or value. I’ve finally added drag-and-drop to the Layers window for moving layers around, along with spiffy animations and transitions (it uses the same code from the newer image thumbnail list at the top of the window). I even added a setting to choose between the Blue theme (from 3.5) and a new Light/white theme (which is the new default). I’ve decided to cut some features in the interest of being able to finish and ship 4.0 this year, such as a plugin manager, but there’s always another train coming (4.1, 4.2, etc.) so I wouldn’t hold a memorial service just yet.

Sort of, anyway. They don’t touch the malware side of the issue at all. I got this in my inbox just now as part of their “Software Publisher Newsletter,” written by their Vice President & General Manager, Sean Murphy:

We are on the verge of fulfilling our vision of coming to market with an installer model that delivers files faster and more efficiently to users, while enabling developers to a) opt-in to the Installer, b) influence the offers tied to their files, c) gain reporting insight into the download funnel, and d) share in the revenue generated by the installer.

The ability to opt-in is not a feature request. It’s a Day 1 feature requirement. Download.com has been making money on other people’s software by barfing malware all over their customer’s systems (and by “their” I’m referring to the software authors).

Consider the Customer Experience Improvement Program (CEIP) that Microsoft has as part of Windows, Office, and numerous other software they release. It gathers information about how you use the software (e.g. which features you used, which buttons you clicked), and reports it to Microsoft. This clearly has enormous privacy implications. But, you have to opt-in to use it. The software asks clearly and politely if you’ll let them do this. The data is recorded in an anonymous manner. Things like permission and anonymity are not “features.” They are requirements.

Oh, and Microsoft actually uses that data to improve the software they release. Bazinga.

He continues:

First, on the press that surfaced yesterday: a developer expressed anger and frustration about our current model and how his file was being bundled. This was a mistake on our part and we apologize to the developer and user communities for the unrest it caused. As a rule, we do not bundle open source software and in addition to taking this developers file out of the installer flow, we have gone in and re-checked all open source files in our catalog.

Ok, whatever Sean. Nobody believes you. You’re not sorry about what you did, you’re sorry that people noticed and cared. You care that the bad press crossed a quantitative threshold, and that it just might affect your quarterly profit reports.

I wanted to verify that they had indeed fixed their catalog of open source software. VLC Player is, in fact, not being packed with their download wrapper (earlier blogs reported it was being bundled). Same goes for GIMP, GIMP Portable, and GTK+ 2 Runtime Environment (which are the first 3 hits if you search for “GIMP”). So from this tiny sample size of 4, it seems he is at least telling the truth for once. (I’d provide links to their pages on download.com, but I’d really rather not contribute to their page rank in any way.)

You can read the whole thing, if you’re really bored, over at their website.

I posted earlier about how download.com was wrapping Paint.NET with their own download manager / installer which would install a toolbar or some other stupid thing. Thankfully this issue was resolved, although not without a few bumps (“yes we removed it” … 24 hours later … “no you didn’t” … 6 hours later … “OH sorry we’ve fixed it” … “damn right you have”).

“AngryTechnician” just tipped me off that now this issue is starting to get even more attention. It looks like they’re even wrapping Nmap, a free Windows utility for doing some kind of network security scan (which I’m not familiar with).

“It is bad enough when software authors include toolbars and other unwanted apps bundled with their software. But having Download.Com insert such things into 3rd party installers is even more insidious. When users find their systems hosed (searches redirected, home pages changed, new hard-to-uninstall toolbars taking up space in their browser) after installing software, they are likely to blame the software authors. But in this case it is entirely Download.com’s fault for infecting the installers! So while Download.Com takes the payment for exploiting their user’s trust and infecting the machines, it is the software authors who wrongly take the blame! Of course it is users who pay the ultimate price of having their systems infected just to make a few bucks for CNET.”

Apparently some antivirus software has gotten on the right side of this issue and are now classifying the CNET download wrapper as malware.

Now, here’s the hilarity. CNET claims that this thing “powers secure downloads”. If you read that post, it’s clear they’re grasping at straws.

“In addition to making our downloads more secure, it has extra features like the ability to pause a download and launch the software installer immediately after it’s finished downloading.”

Whoa there … I can pause downloads?! Stop the presses!

Look, CNET, this is not a great monetization tool for your partners. It’s a fucking virus and you’re trashing the goodwill and reputation of everyone around you in order to make a buck. One thing I realized early on is that is not how you build a career (it’s one reason why I refuse to do any type of bundling). This is not an exciting opportunity. Look at it this way: while you’re at the bar, sharing a drink with friends, and they’re talking about something cool they did or a tricky technical problem they solved at work, you are going to keep your mouth shut. Because you’re ashamed of your job.

For continued hilarity, check out the title of this blog post. O rly? “Be careful when downloading software?” Does that advice include editing your hosts override file so that download.com redirects to disney.com?

Paint.NET’s performance is an interesting topic. There are some areas, such as startup, where Paint.NET is very fast. There are other areas, such as selection manipulation, where it is quite slow, even laughably so in non-pathological situations. If you’ve ever worked with the Magic Wand tool on an image of more than about 4 megapixels, then you probably understand what I’m talking about.

Just about the slowest thing you can do in Paint.NET v3.5.x is to use the Magic Wand on a large image and then try to move, rotate, or combine that selection. The geometry processing is slow, and the rendering takes just as long. I even ran into a test case where clicking the Magic Wand on a very high resolution image of the Milky Way took hours of processing time and a few gigabytes of memory. Ouch. (I actually killed the process after about 20 minutes, so “hours” is just an estimate.)

At this point I’m going to jump ahead to the results of what I’ve done. I took* a 6 MP image (3871 x 1595 pixels) of a 2011 BMW 5-series and loaded it up in v3.36, v3.5.10 and in 4.0. (For those who aren’t aware, 3.36 and 3.5.x are basically two separate generations of the Paint.NET codebase.) I then resized this image to be 5x larger, giving me a 136 MP image (19,355 x 7,975). I clicked with the Magic Wand tool in the same spot in each, waited for the selection to be created, and then did some simple things like scrolling, zooming, and drawing a gradient (which is clipped to the selection).

In 3.5.10, scrolling and zooming were very slow, limited by the rate at which the geometry of the selection could be continually rasterized. Drawing something like a gradient was able to run at about 1 frame every 2 seconds. The app just couldn’t deal with it. CPU usage was extremely high the whole time.

In 4.0, scrolling and zooming were perfectly fluid. Oftentimes the selection itself would lag behind what I was doing but it always caught up a moment later. It didn’t get in the way of me trying to find the area of the image that I wanted to do stuff with next. I could even draw a gradient, clipped to the selection, and it was responsive the whole time (the status bar and whatnot kept updating). Now, to be honest, the gradient actually takes longer to render than in 3.5.10, but it fills in gradually as each tile completes its work, and allows the user to keep doing other stuff while it’s in-progress. This is a much better situation to be in: I can and will optimize the gradient renderer to be faster at some point, but in the meantime I still have something I can ship and that people can get utility from.

If we go back to Paint.NET v3.36, the situation becomes almost comical. You see, 3.36 had an animated “dancing ants” selection outline, and it uses GDI+ to render this on the UI thread. As such, in this test case, it is constantly chewing up CPU while also providing an almost entirely non-responsive UI. You can’t even hover the mouse over the menus to get them to light up and indicate they can be clicked without waiting a second or two first. The 3.5 release improved this by removing the animation. 4.0 will give us the best of both worlds: we get the animation, and we also get trivial CPU usage. We also get all sorts of other improvements.

The way I do this in 4.0 is to render the selection on another thread. Actually, I render masks, which are 1-byte per pixel bitmaps that only store the alpha channel. Brushes are then used to fill in the color. I create 5 of these masks: 1 for the interior, and then 4 for the outline to take care of the animation. When I go to draw the selection back on the UI thread, I just use ID2D1RenderTarget::FillOpacityMask() three times. Once for the interior, once for the Nth mask with a black brush, and once more for the ((N+4)%4)th mask with a white brush. This increases memory usage and GPU fill rate requirements, but with dramatic improvements to responsiveness, CPU usage, and battery life.

Because I’m using Direct2D and do this mask filling with hardware acceleration, CPU usage for an animated selection almost non-existent. Click on the screenshot below for evidence.

paint.net 4.0 sipping on CPU usage while still animating a very complicated selection outline

In summary, I’ve traded 1.25 frame buffers of memory for a huge performance win. Manipulating selections is still just as slow as it ever was, and over time I plan to move that work off the UI thread or whatever it takes. Right now if you use the Move Selection tool on a complex selection, it does a bunch of matrix math to transform all the points before committing the changes to the Selection object. Well, clearly we can render that in a much easier way: take the cached bitmap and just apply the same transformation. etc. etc. One step at a time.

* By “take” I don’t mean that I was the photographer.

One nice thing about the Move tools in Paint.NET is that they implement something I call fine-grained history. When you go to edit a selection or move its pixels around, you get 1 history entry for each adjustment you make. This allows you to move it around, rotate it, etc. and then when you press “Undo” it only backs out of the most recent adjustment you made. Contrast this to the Gradient, Line/Curve, and Text tools which don’t keep track of each edit you make and can only undo things wholesale.

Recently I’ve been working on adding this to the Gradient tool for 4.0. This support will then make its way into the shape tools, and probably the Text tool as well (also, shapes will be undergoing an overhaul soon). The reason I’ve been working on the Gradient tool is that it is a fairly simple tool and this allows me to get the code for fine-grained history working without having to worry about as many “moving parts.”

Here’s a screenshot showing the Gradient tool having gone through it’s initial “draw” operation, followed by two “edit” operations:

(click for full resolution version)

Once this makes its way to the new Shapes tool (not yet written unfortunately), you’ll be able to: 1) draw a shape, 2) move, rotate, resize the shape, 3) change the colors, 4) change the stroke, 5) change any other properties, and then 6) finish/commit it to the layer. Currently the shape tools in v3.5 are fairly barbaric and don’t even let you change their color or other properties once you’ve pressed down the mouse button. It can be quite painful if you are consistently off by a few pixels and have to constantly redo your Rectangle until it’s just right.

There are some other important benefits to doing this. I’ll be able to significantly reduce memory usage, for starters. In Paint.NET v3.5, the tools are implemented in a rather cumbersome way. While you draw, it’s modifying the layer data so it contains what you’re drawing … then it draws that to the screen. Then, when you move the mouse it has to copy back the original contents of the layer before drawing the new stuff. When you’re done, it has to copy back the original pixels, save it to disk, then redraw everything one last time. This is implemented with a very carefully debugged protocol that requires all sorts of data to be copied around various buffers before, during, and after any edit operation. Any bug in the protocol for drawing, copying, and emitting history entries can lead to data loss.

With the new system, the gradient (et. al.) draws itself using a layer overlay, which only affects the rendering pipeline and not the image itself. Undoing the edit before it’s committed is a simple matter of throwing away the overlay. When it’s time to commit the drawing, the code just says “hey I changed this rectangular area” and then a simple diffing algorithm copies away only the parts of the layer that changed. There’s no need for me to manage complicated geometry masks and dirty regions any more (other than what’s needed for an active selection, of course). All the large and temporary (and permanent, ack!) buffers are simply no longer needed.

This is all very important! Reducing memory usage is very important for an application like Paint.NET, especially on 32-bit systems. This also removes another barrier toward migrating to a fully tile-based memory allocator, which then opens up doors for many other very, very cool features.

Another benefit to doing this is that it moves the tool rendering code off of the UI thread and into the rendering pipeline (which was already moved off the UI thread for 4.0). The UI will always be responsive while you’re drawing, even if the canvas is still “catching up” to what you’ve done. It’s pretty cool to be able to throw around the Gradient tool’s “handles” (aka “nubs”) and see them responding at a much higher framerate than the canvas itself. If you’ve ever had to work with an image that is very large, or has a lot of layers, or both, then you probably already know that this will be a great thing.

Also, right now the Move Selected Pixels tool is a monster (for me that is; not for you). While it does work perfectly well, the code is wildly complicated and fragile. Adding or changing any features would be a recipe for disaster. By migrating it to this much simpler system, I’ll be able to reduce memory usage further, increase performance and quality, and remove any “hitch” you may notice at the start of a move operation. The reason the “hitch” happens is that the history data (the layer’s pixels) must be saved off into a temporary buffer before any rendering can be done. Afterward, there’s a frightening dance of copying pixel data from the layer to a scratch area, and then transforming pixels from a 2nd copy back to the layer, and then undoing it all before redoing it, etc. The nightmare gets worse when history entries come into the mix. The code is basically the worst time travel paradox you could possibly imagine – I can’t honestly say I fully understand how it works, even though I wrote it. What I can say is that it works, but only because it’s been thoroughly and exhaustively tested, debugged, and bugfixed.

Oh, and you’ll notice in this screenshot that I’ve finished up the last bits of UI reorganizing. I’ve done away with the Window and Utilities menus. Instead, each of the floating windows has its own toggle button at the top-right. The gear icon opens up the Settings dialog, and the question mark icon gives you the traditional help menu. Anything that was in the Utilities menu has either been moved into the Settings dialog, or removed. I‘ve also increased the size of these icons to 24×24 because I found they were just too darn small at 16×16.

Within the Settings dialog I’ve added a Diagnostics page which gives you a peek at your system information as Paint.NET sees it (basically the information that’s printed at the top of a crash log, and more). It will also give you access to the crash logs folder and some other simple things that should make troubleshooting a bit simpler. Oh, and for those who don’t like the “dancing ants” (animated selection outline), I’ve included a checkbox to control whether this is on or off (not on the Diagnostics tab though).

In time, I hope to elaborate further on what I mean by “this stuff opening doors for other very cool features.” I’ve been throwing around ideas for the last few days and it’s exciting how much things are radically simplified by all this. I’m hoping that 4.0 will serve as a solid basis for many, many features over its lifetime. 4.0 won’t have all the features everyone wants, but it will enable me to finally add so many more of them!

Update: (October 4th, 4:30pm PST) Download.com is no longer doing this (at least, not with Paint.NET).

I just noticed that download.com updated their listing of Paint.NET to version 3.5.9. Neat. Except that now they’re wrapping it in their download manager which tries to get you to install the “StartNow” toolbar. What a bunch of **** (insert your favorite 4-letter word).

I’ve already sent them an e-mail telling them that this is a violation of the Paint.NET license and to either cease this, or to remove Paint.NET from their site.

In the meantime, I obviously recommend not downloading Paint.NET from download.com.

More info on this despicable practice: Download.com wraps downloads in bloatware, lies about motivations (extremetech.com)

Update: (October 4th, 9:30am PST) I received a reply from them at 2am stating they had removed the download manager from their listing of Paint.NET v3.5.9. However, as of 9am, they have not actually done this. If you go to the Paint.NET v3.5.9 listing on download.com, it still has their spammy download manager attached to it!