Below is a brief overview of what was added during the year:

• View Filter feature, enabling users to restrict viewing to, say, images with a 5-star rating, those marked for deletion, or those whose file name matches a given pattern.
• IPTC Metadata Editor plug-in, with ability to write IPTC fields like headline, caption, location, copyright, with spell-checking, either within files (JPEG, TIFF, HD Photo) or in external XMP sidecars. The IPTC Editor also features a dictionary-based keyword entry with auto-suggest and search-as-you-type. The program supports pre-defined keyword hierarchies such as the Controlled Vocabulary Keyword Collection.
• Comprehensive support for GPS Metadata, with GPS Location displayed on-screen, ability to launch Google Maps, Bing Maps, Flickr or Panoramio on an image’s GPS coordinates, sorting on GPS timestamps, GPS-related selection criteria’s for batch file processing in the File Utilities plug-in, including proximity matching, direction, altitude, speed and more, as well as reverse-geocoding of GPS coordinates in the IPTC Editor.
• Raw compatibility has been extended to support dozens of new camera models from all major manufacturers. In parallel, the optional FastPictureViewer Codec Pack add-in was enhanced to support Photoshop PSD files and CGI formats like DDS, SGI, Softimage PIC, OpenEXR, MPO, JPS, Maya IFF, VTF and more.
• New RGB Eyedropper feature with real-time 1:1 zoom window and shadow/highlight indicators.
• New FTP Upload action in the batch file processor, that can transfer files to a remote internet server using the File Transfer Protocol. Error control and activity logging have also be added to the File Utilities batch processor plug-in.
• Several smaller features have been added (random slideshows, selectable background color, new keyboard shortcuts…) and of course numerous bug fixes and performance enhancements through the entire product.

All in all, more than 50 update releases with fixes and new features were made over the course of 2011, with more to come next year!

The first questions that come to mind are is it any good? Is my camera and my operating system both supported by this new package? How well does it run compared to other similar products?Immediately after the release of the Microsoft Camera Codec Pack we began receiving emails from our users asking for comparisons between our own FastPictureViewer Codec Pack, a mature product with 2.5+ years in the field, and Microsoft’s new offering.

First thing first, the Microsoft Camera Codec Pack is a free download from the Microsoft website. The FastPictureViewer Codec Pack, on the other hand, is a commercial product that can be licensed from us for a fee. That makes for our first point of comparison.

Microsoft pretends having “changed the game” regarding raw files handling in the Windows Operating System and achieved “GPU-like” performance by taking advantage of multicore CPUs, as well as covering all grounds regarding platform support and camera models, past and present. July 2011 YouTube video here.

So far so good, the commercials sound vibrant but conspicuously omit a couple of key facts:

Raw support in Windows is nothing new

On June 1, 2005, Microsoft officially announced their intention to deliver raw support to Windows Vista “that will enable consumers to easily work with RAW files” (press release).

Later the same year, the company released the Microsoft RAW Image Thumbnailer and Viewer for Windows XP, adding raw support for a number of digital cameras to their flagship operating system of the times.

Finally, Windows Imaging Component (WIC), the framework upon which WIC raw codecs rest, shipped as an integral part of the operating system with Windows Vista and XP SP3 in late 2006 and was part of every Windows releases since then.

Camera manufacturers and 3^rd parties have been supplying codecs for years

Canon, Nikon, Sony, Olympus, Pentax, Panasonic and small independent software vendors like us FastPictureViewer and Ardfry Imaging have been supplying raw codecs for Windows for a number of years (as early as 2006 for some). It took Microsoft more than 6 years to come up with their own take, but partners and ISVs have been supplying Windows-compatible codecs based on the WIC infrastructure for a long time.

Back to our subject.

The first thing we’d like to mention is that, while raw support is indeed a very nice thing to have in any operating system, some basic features like automatic rotation of plain old JPEG files, according to camera orientation, are still not addressed by Microsoft as of today: portrait images are displayed sideways in Windows Explorer, Windows Photo Viewer, Windows Photo Gallery and Windows Media Center, unless manually rotated.

The FastPictureViewer Codec Pack addresses this shortcoming by providing a JPEG decoder with the intelligence needed to recognize the orientation data written within the files by digital cameras and the muscle to do the rotation when required.

When our JPEG codec detects that an image needs to be rotated it performs the necessary action on-the-fly, without ever modifying the original file, and Explorer, Photo Viewer, Photo Gallery and Media Center 6.1 auto-magically starts displaying images in the correct orientation!

JPEG Decoding Performances

This brings us to our first performance metric: JPEG has been around for a long time and JPEG decoding libraries have been optimized and re-optimized over the years. One can expect that the JPEG decoder created by Microsoft and shipping with every copy of Windows since 2006 (Vista and XP SP3) is very well optimized – and in fact it is.

Still, we managed to produce a JPEG codec that runs quicker than Microsoft’s, despite the additional work required to perform the rotations:

Time required to decompress (and rotate, in the FastPictureViewer Codec Pack case) an out-of-camera portrait-oriented 12.1MP JPEG image created by a Nikon D3 DSLR. Intel i7-970 processor, times in milliseconds, lower numbers means faster performances. Results averaged over 5 consecutive runs with the same image after 1 priming run to factor out disk performance.

That’s right: our auto-rotating JPEG decoder performs additional work and still runs 25% faster, rotating an image, than Microsoft’s which simply ignore the rotation information.

Now that the introductions are made, we can dig through the heart of our subject.

Supported Camera Models

Number of camera models supported by each codec pack. Higher number is obviously better.

While Microsoft pretends to have covered all grounds they are far from supporting all raw-capable cameras on the market. Perhaps more importantly, their support is frozen 18 months back (at the time of this writing, and counting), probably due to exceedingly long internal testing phases and sign-off procedures.

That means that all cameras released in the past year and a half are not supported by the Microsoft Camera Codec Pack, this includes cameras like the Canon 60D, Nikon D7000, Nikon D3100, Nikon D5100, Sony NEX-5 and many, many others that can be seen today on every retailer’s shelves.

The 18 months gap gives a glimpse at their release cycles, which are unlikely to get any shorter anytime soon (the original Microsoft RAW Thumbnailer and Viewer for Windows XP released back in 2006 was never updated with new models support in 6+ years).

By contrast, the FastPictureViewer Codec Pack gets refreshed about 6 times a year with raw compatibility updates, adding new features and timely support for all the latest models on the market: for example support for the Canon 60D and the Nikon D7000 – both missing from Microsoft’s offering – was added to the FastPictureViewer Codec Pack on November 11, 2010 already, and we have added more than 20 other new models since then including NEX-3, NEX-5, D5100, A230, A290, 600D, T3i, X5, 1100D, A390, X100 and more, raising our total to 379 387 supported models as of August October 2011 vs. 120 for Microsoft.

Microsoft has a long gap to fill regarding camera model support and will likely always be late to the game, due to their lengthily release cycles and everything else we don’t know about (such as changes in internal priorities). Writing imaging codecs requires a very specific skillset and chances are that the people able to actually do this efficiently are few and far apart, and likely have better (more important) things to do… Time will tell.

Platform Support

The Microsoft Camera Codec Pack does not support Windows XP nor Windows XP 64 nor Windows Server 2003. It does not support Windows Vista RTM nor Windows Vista SP1.

Windows XP alone still has about 50% global market share at the time of this writing, if we believe the operating system statistics supplied by specialized companies. That’s a lot of users left out of the “holistic story that covers all Windows platforms” recently touted by Microsoft.

By contrast, the FastPictureViewer Codec Pack supports all platforms onto which Windows Imaging Component can be (or is) installed. Our product can be installed on Windows version going as far back as Windows XP SP2 and Windows XP 64. Of course functionality is reduced on the old platforms (the XP Fax viewer will never display any raw files) but we provide extremely quick thumbnail support to Windows XP and XP-64 for all the camera models and file formats handled by our FastPictureViewer Codec Pack.

Raw Previewing and Full Raw Decoding Performances

When Microsoft originally introduced the Windows Imaging Component platform back in 2005, on which the codecs discussed here rests, it was told that capabilities would be added to the platform to allow for controlling the raw conversion in such a way that makes it possible to build raw converter applications on top of the new codec-based infrastructure.

As of today there are no (zero) known application that takes advantage of these features and as a matter of fact, the just released Microsoft Camera Codec Pack returns a “not implemented” error when queried about its raw conversion control abilities, via the QueryRawCapabilitiesInfo() call, which means that all the Microsoft codecs have to offer today is a default raw conversion using arbitrary internal parameters, with no user intervention possible.

If one compares this complete lack of “tunability” to the level of fine control exposed by, say, Adobe Camera Raw, Adobe Lightroom, RawTherapee or Bibble Pro, just to name a few serious raw converters available on the market today, or open source efforts like UFRaw or DCRaw, one readily understand that WIC raw codecs are not going to have any profound impact on the “real” raw conversion business landscape anytime soon, where users tweak every minute aspects of the raw conversion process using countless sliders, presets, camera profiles and even individual lens profiles.

As a matter of fact, Thomas Knoll (the original author and lead architect of Adobe Photoshop and Adobe Camera Raw, the raw conversion engine used by Adobe Lightroom and Photoshop) described WIC raw codecs as “basically useless for Camera Raw / Lightroom type of applications” (read his own words in Adobe’s forum).

If one is looking for full photographic control, “a WIC codec is likely to prove unsatisfying”, according to Zalman Stern, Tech Lead for Adobe Camera Raw and author of the Adobe DNG WIC Codec (read his post on DPReview).

This leaves us with one type of use for WIC raw codecs, albeit a very important one: help users visually organize their files in Windows Explorer by providing high quality thumbnails, fast previews as well as quick slideshows in Windows Photo Viewer and Photo Gallery.

Users of Windows Media Center on Windows 7 or later also benefit from having WIC raw codecs installed as they can project raw slideshows on their living room TV though a Media Center Extender, such as the Xbox 360: raw support at the platform level is great!

Most camera manufacturers had the good idea to include a ready-made preview image within their raw files. This image is usually stored in compressed JPEG format and reflects all camera settings such as sharpening, contrast, white balance, and manufacturer “secret sauce” like Nikon Picture Control or Canon’s Picture Style. In fact, all camera options available through the camera’s menus and modes are reflected on the preview image, which is made by the camera itself and stored in the raw files as the pictures are taken.

This preview image is perfect for the uses mentioned above: JPEG data can be unpacked very quickly (we showed above that we are pretty good at this) and the resulting image is faithful to what the photographer saw with its own eyes when the image was captured: if you like Canon’s skin tones then you are going to get them on the preview JPEG, if you like Nikon’s sunset colors you’ll get them on the previews too. In fact all the coveted image science of the camera manufacturers is reflected on these preview images, unlike raw conversion from non-manufacturer software which basically throws away all this goodness and starts over from scratch.

We realized this long time ago and the FastPictureViewer Codec Pack is optimized for the previewing, slideshow and thumbnail scenarios described above, probably covering 99.x% of all WIC raw codecs uses so far. Our codecs use the JPEG images by default and provide stellar previewing performances, letting applications display a high-quality previews far quickler than when converting the full raw data, with faithful colors and all camera settings applied. Edits made to the raw files with manufacturer software (such as Nikon’s Capture NX) are also reflected on the preview image and shown by our codecs on the thumbnails, Explorer’s preview pane and full-scale images.

On the contrary, Microsoft, probably pursuing different goals, elected to ignore the preview image entirely – it’s not even exposed by their codecs -and instead concentrated all its efforts to speed up the full raw conversions.

As a matter of fact they did an impressive job but without a doubt the resulting images invariably differs from the camera produced JPEGs, sometimes in subtle ways and some other times in not so subtle ways, with obvious color shifts or exposure errors (remember: there is no way to tweak the conversion process so you have to live with what they give you, and no camera effects are applied!)

As written above, manufacturer and 3rd-party applications like Capture NX or Adobe Lightroom update the preview image in NEFs and DNGs, respectively, after edits are made. The FastPictureViewer Codec Pack’s use of the preview image means that those edits will be visible on thumbnails and Explorer previews with our product, but the Microsoft package, always starting over from the original raw data, will not show any changes made by users.

As far as performance is concerned, we picked files from a couple of flagship camera models from Canon and Nikon, as well as a few mid-range models and timed both the full raw conversion performances of both packages, as well as the FastPictureViewer Codec Pack’s unique “fast previewing” optimized mode, which is active by default (our codec’s behavior regarding the use of the embedded JPEG preview can be adjusted from a supplied Control Panel applet, so if you insist for full raw conversion you can have it too).

Here is what we came up with (remember: the Microsoft product does not support any new camera, we had to stick to old models so no 60D or D7000 numbers, sorry).

Nikon NEF (D3X, D3, D300S, D2X, D70S)

Nikon NEF full raw decoding performance of both Codec Packs, as well as FastPictureViewer's unique "fast preview" optimized mode, shown in green with actual time on top. Intel i7-970 processor, times in milliseconds, lower numbers means faster performances. Results averaged over 5 consecutive runs with the same image after 1 priming run to factor out disk performance.

Canon CR2 (EOS 1D MkIV, 7D, 5DMkII, T2i, 1000D)

Canon CR2 full raw decoding performance of both Codec Packs, as well as FastPictureViewer's unique "fast preview" optimized mode, shown in green with actual time on top. Intel i7-970 processor, times in milliseconds, lower numbers means faster performances. Results averaged over 5 consecutive runs with the same image after 1 priming run to factor out disk performance.

As can be seen on the above graphs, Microsoft’s efforts so far regarding full raw conversion approximately matches the FastPictureViewer Codec Pack’s for the NEF and CR2 formats we tested when used in full conversion mode, within a few percentage points. Five times out of ten, the FastPictureViewer codec is slightly faster than Microsoft’s “Multicore, GPU-like performance”, and slightly slower the other five times.

What is striking, however, is how well our unique Fast Preview mode performs and how much quicker it is compared to both package’s full raw decoding scores.

In extreme cases, such as the current top Canon EOS 1D MkIV and EOS 7D models, the fast previewing – which creates a full-resolution 16MP image for these camera, is more than TEN TIMES quicker than the full raw conversion counterpart.

In the case of the lower-end Canon 1000D, which only stores a medium-resolution preview, the performance difference is even greater, from Microsoft’s 700ms best time (0.7s) down to 20ms (0.02s) for our Fast Preview time, a shockingly unreal THIRTY FIVE times (35x) quicker.

The performance gap is not as big with the Nikon NEF files we tested, still, we observed performance difference in the order of four to ten times faster in favor of our product when using the default Fast Previewing mode, and comparable performances in full raw mode.

All Nikon cameras since the D2 store a full-resolution preview image, so are new cameras from Canon and most other manufacturers, so there is little incentive to decode the full raw data just to produce a thumbnail or screen-size preview from it.

The images returned by the FastPictureViewer codecs in Fast Preview mode are probably not suitable for “darkroom quality” art gallery printing (neither are Microsoft’s full raw conversions anyway) but the point is no one is going to use this type of codecs for true art-quality work, if only for the lack of control over the conversion process.

The quality of images produced by the FastPictureViewer Codec Pack is undistinguishable from camera-produced JPEGs (in fact, we actually use a camera-produced JPEG in fast preview mode) and the resulting image quality and decoding performances are perfect for the codec’s most likely use today, which is mainly to provide thumbnails and instant previews to the user, to facilitate raw file management in Explorer.

To conclude the performance chapter, Microsoft’s full raw performance is indeed much higher than manufacturer-provided codecs but is basically on par with the FastPictureViewer Codec Pack in the tests we made, and then we provide an even faster Preview mode that effectively makes Microsoft’s and all manufacturer codecs appear dead-slow.

In all fairness, we must say that both the FastPictureViewer Codec Pack and the Microsoft Camera Codec Pack are in a completely different league than most manufacturer-produced codecs with regard to raw decoding performance, so Microsoft can rightfully claim that their package performs very well. So are we.

Of course, in real scenarios, factors such as disk speed and fragmentation and the fact that thumbnails, when created, are written to a database which often resides on the same disk as the one containing the images, as well as variable factors such as computer load, available CPU and memory resources etc will tend to limit the actual performance as observed from the end-user but still, waiting one or two seconds for a file to open, or one or two tenths of a second makes a tremendous difference in the user experience and in the perceived quality of the platform – Windows – compared to competing platforms such as the Apple Mac or – heaven forbids – Linux.

We believe we do a better job at this with our Codec Pack than Microsoft with theirs, and an even better job than camera manufacturers who apparently missed a thing or two in Programming 101, for the most part supplying codecs producing very good quality images but with abyssimal performances and sketchy features (check the level of user satisfaction on Canon’s forum, for example).

Keep in mind that we used one of the fastest CPU available for these tests: a six-cores Intel i7-970 with 12MB of cache and 12GB of memory. On a slower CPU all numbers should grow proportionally.

Metadata Exposed in Windows Explorer

One important aspect of being able to effectively organize file in Windows Explorer is the ability to examine annotations, rating, camera and exposure information and other data stored in the files, either by the camera itself or by manufacturer software such as Canon DPP or Nikon Capture NX. The FastPictureViewer Codec Pack goes through great lengths to expose as much of this data as possible, first to Windows Explorer but also to the Windows Search Indexer and all other interested parties.

See the result below:

Metadata surfaced by the Microsoft Camera Codec Pack

Metadata surfaced by the Microsoft Camera Codec Pack from and out-of-camera Canon 7D CR2 raw file.

Metadata surfaced by the FastPictureViewer Codec Pack:

Metadata surfaced by the FastPictureViewer Codec Pack from and out-of-camera Canon 7D CR2 raw file.

A picture is worth a thousand words regarding metadata: we just expose far more of it than Microsoft currently does (about 26 vs. about 10 elements).

How we Tested

Evaluating codecs is more difficult that testing normal applications. There is no user interface and it is actually difficult to be certain that a particular codec is in use and not another. To avoid any confusion, we always uninstalled a package then rebooted the computer then installed the other package and rebooted again.

This way, we made absolutely sure that only one of the two packages was installed at any given time, and we knew which one beyond any doubt.

We disabled the Search Indexer as it immediately starts crawling as soon as a new codec is installed, interfering with performance tests as it exercises the disk substantially. We also let the computer settle down for a few minutes as a lot of activity occurs after startup.

We used a tool from Microsoft called WIC Explorer to load the images. This tool displays the load time in milliseconds, then we repeatedly Unloaded then Loaded the test images, writing down the times we got for 6 consecutive runs, then we averaged the last five runs. The timing was exclusively provided by this tool, there is no human lag in the measurements.

The first run was never taken into account as file loading time (disk activity) is unpredictable. By skipping the first run we ensured that the disk cache was primed and that we tested the codecs and not the disk subsystem. Any overhead induced by the WIC Explorer tool itself is included all measurements made, so there is some constant ‘C’ added to all times shown.

We tried to use the Real Time Performance Test of another Microsoft-supplied tool called WIC Cop. As an anecdote, the “not implemented” error returned by the Microsoft Camera Codec Pack, when queried for its raw capabilities, caused the codec performance tool to crash. Yes, you got that right but please let us repeat slowly: Microsoft’s raw codecs causes Microsoft’s raw codec performance test tool to crash – go figure!

Wrapup and Conclusion

The FastPictureViewer Codec Pack supports more Windows platforms, far more camera models (3x more and counting) and more metadata elements than the Microsoft Camera Codec Pack. It also provides far better performance for the typical uses of this type of codecs, as well as comparably fast performances for popular raw formats in the rare occasions where full raw conversions are desired.

Our product is likely to be updated far more often than Microsoft’s (as a matter of fact, we made two small releases since they released their product already, with a 3rd refresh currently being tested as we write this).

Our turnaround is in days not years, and we’ll always be more agile than any huge corporation. We’re actually willing to bet a FastPictureViewer Codec Pack license that the folks appearing on the Microsoft videos and posting to corporate blogs about the greatness of the Microsoft Camera Codec Pack don’t even know who, inside the company, actually produced it. When you send us an email, it is read by the very person responsible for the architecture, design and implementation of the product. Check this story from one of our users, aptly titled Outstanding responsiveness from the FastPictureViewer codec team!

From a “big picture” perspective it can only be good that more people get their hands at raw codecs, enabling them to see their files as images at last. What is not so good is that Microsoft elected to not even mention the existence of first paty (manufacturer) as well as 3rd party codecs like the FastPictureViewer Codec Pack or other WIC codec providers such as Ardfry Imaging.

It is also regrettable that they removed the “Pro Photo” website where anyone could find out about available codecs, among other things. Instead they play it like “we added raw support to Windows just now” and made sure their “single-click” download process goes directly to their Download Center only, effectively making us and other suppliers invisible.

The end-result is less choices for consumers, Microsoft quickly establishing itself as the only game in town in yet another tiny segment, putting small partners at risk of going out of business for no good reason: it is not like we were competing against them, more like helping them quench the exodus of photographers to the Mac and Linux platforms…

Worth Mentioning

Our product does not stop at 379 camera models, we also provide very fast support for Adobe PSD documents, with thumbnails, metadata and previews, as well as support for about a dozen image formats used in the Computer Graphics and Computer Game industries (EXR, HDR, PIC, SGI, IFF, TGA, DDS and several more), support for 3D digital cameras (MPO, JPS), thumbnail and preview extraction from Encapsulated PostScript (EPS) files, Adobe InDesign INDD and Adobe Illustrator AI files, thumbnail extraction from PDF files containing an XMP metadata packet (i.e. PDFs created by Adobe Acrobat software, mainly), plus a few special formats like Adobe Lightroom’s preview files.

We support more than 45 file extensions in total in the Codec Pack (no bloat: all our codecs are user-selectable at installation time), with more formats and camera models being added all the time: check the release log on the Codec Pack’s web page!

Support us!

Like this page, tweet it and retweet it, link to it from your blog or website, +1 our web pages and tell all your friends: there is hope for timely and fast Camera Raw support in Windows, and it does not come only from Microsoft…

Every bit helps and every vote counts so act now! (and don’t forget to grab our free trial)

– Axel Rietschin (founder and author)

These container formats were designed to be extensible in the sense that it is possible to define new content parts by creating new tags, and this extensibility worked well already as it allowed adding EXIF data to JPEG images when EXIF was invented, for example, without the need to significantly alter the JFIF container format specifications.

However, when it comes to the physical implementation, the files are laid out as a set of contiguous segments, and adding new metadata elements to an existing file is not possible.

Let’s say an image does not contain copyright information and an application is used to add this information afterwards. With the current file layouts, the application needs to extend the size of the EXIF segment in order to make room for the additional data. The larger segment will no longer fit at its original place in the file, and the application needs to build an entirely new file, deconstructing the original file by splitting it into its various constituents and building a new file from scratch, moving file segments one by one in the correct order, and of course substituting the original EXIF data segment with the new, lengthier one.

One unfortunate side effect of this “transcoding” operation is that it shifts parts of the file further back to make room for the additional data at the front, thus the new file is substantially different from the original and all internal offsets must be adjusted accordingly. (“Offsets” are values indicating the location of various elements in the files, written at file creation and used by file parsers to find their way through the various file constituents).

Camera manufacturers insist to store opaque data segments in their files. Those segments, known as Maker Notes, are present in JPEG, TIFF and raw files created by all digital cameras and contain undocumented proprietary information – you can think of it as private metadata.

The problem is that 3rd party applications does not really understand the content and meaning of this private data and cannot “fix” it appropriately when reconstructing files with an altered layout.

As a result, more often than not, files that were transcoded to make room for additional metadata are left in subtly broken state: while whey may appear to work well with some applications (in particular, with the 3rd party application that edited them), manufacturer software may fail to recognize the files afterwards, or may fail to be able to correctly interpret the private data they contain since the offsets still refers to the original file layout that was created by the camera’s firmware, and not to the layout reconstructed by the application that added the extra metadata. One exception to this is the manufacturer’s own software, one example of which being Nikon’s Capture NX, which knows all details about Nikon’s proprietary files and can of course rewrite them properly.

There is no easy solution to this problem. Adobe’s DNG effort, for example, does not help at all since the Maker Notes are stored as-is in DNG, and are just as opaque as they were in the original file.

Around 1996, Microsoft came up with an attempt whose intent was to mitigate the problem: they created the OffsetSchema tag (59933), providing a well-known place where an application would write the amount of bytes the Maker Notes were moved in the new file with regard to their original location within the original file.

The idea was that file readers who actually understand the Maker Notes (i.e. the manufacturer software) would have a chance to repair the internal offsets by looking at the OffsetSchema tag and, knowing how far the data was moved, be able to fix any opaque pointers by the correct amount and restore the meaning of the moved data.

The creation of this field got some bad press initially, people accusing Microsoft of subverting established standards while their intention was simply to provide a reasonable solution to an existing problem they did not create in the first place.

This attempt was not without problems, however: the main one was that it required a worldwide coordination from all imaging software vendors who’d create (or appropriately update) the OffsetSchema tag when transcoding files.

This never happened and as a result the new tag is not always present in transcoded files and, when it is, its content cannot be relied upon as the files may have been subsequently modified and rewritten by any number of other applications that failed to update it (the default behavior in the presence of unknown tags is to ignore their content and copy them as-is).

To make a long story short, adding metadata to a file requires rewriting it entirely to make room for the new data and often breaks part of the files content in subtle and undetectable ways that can never be reliably repaired afterwards.

During the first transcoding of a file, smart imaging application adds a certain amount of padding data at the appropriate places, right at the end of the metadata segments, leaving some empty space in the otherwise compact file layout to allow some further growth of the metadata without forcing the need to rewrite the entire file again for each update.

Not all application adds this padding data while transcoding files and not all applications take advantage of it to perform in-place metadata edits when the padding room would be sufficient to accommodate the additional data.

The padding also needs to be properly managed, for example if an application adds 1KB of metadata and finds that 4KB of padding exists in the file, it must update the amount of remaining padding to be 3KB. Also, shall the available padding be too little to accommodate the new metadata, an entirely new file must be created yet again and a full complement of fresh padding must be added to the new file, in order to allow fast in-place editing for the next few updates. Unfortunately, current digital cameras never add padding, forcing file rewrites when additional metadata is first added to out-of-camera files later on.

One could think that it would suffice to somehow append the data at the end of the file but the current container standards requires that tags are stored in ascending numerical order, so the EXIF segment must stay at the same logical place in the file and cannot be relocated to the end. Such a scheme could also create issues as multiple versions of the metadata would be present in the files and, for one or the other reasons just mentioned, would likely break many existing software applications.

Another little known issue is that many raw converters (at least all those based on the popular DCRAW source code from David Coffin) rely on the actual file size to identify certain raw files or camera models. Increasing the file size even a single byte would break this detection scheme and pose problems to many existing raw conversion software.

Finally, rewriting a 25MB raw file (or a 80MB TIFF) just to add 1KB of copyright information is inefficient at best and also pose problems with compressed formats as the compressed parts must be moved losslessly from the original file to the transcoded one: this may or may not be easy depending on the actual file format and the availability (and features) of decoding libraries and file parsers handling this particular format.

From the above, one readily understand that a lot of highly complex file surgery happens under the hood when a user hits the ‘5’ key in FastPictureViewer Pro to give a five stars rating to his favorite photo, or when clicking the Save Metadata to File menu option in Adobe Lightroom to export the metadata to a JPEG, most of which could be avoided if the file containers allowed in-place edits…

A big part of what we do here at FastPictureViewer.com is to read image files. We support hundreds of digital camera models and dozens of raster formats in our image viewer and Windows codecs, we also expose EXIF, XMP and IPTC metadata to the Windows Property System, from where it can be consumed by Windows Explorer or any application that cares to take advantage of this data. We also support EXIF, XMP and IPTC metadata embedding within JPEG, TIFF and HD Photo as well as reading and writing external Adobe XMP sidecar files, just to say we are at the heart of all those issues and have been dealing with them, or working around them, on a daily basis for many years.

What we propose below is not a revolution, but a possible step towards easier to manage image files.

Camera manufacturer will probably never give up their competitive advantage and are likely to keep storing opaque proprietary data in their files for the foreseeable future, so the “opaque metadata problem” is unlikely to be solved anytime soon (in an ideal world, opaque binary metadata blocks would not exist and all camera manufacturers would use properly labeled discrete fields to store all the data they need to store. Each manufacturer would be able to keep their trade secrets as there would be no obligation or need to reveal the intent and purpose of each data element, but storing them in discrete fields – say in XML CDATA format – would make it possible for all 3rd parties to deal with all metadata – dissect it, extend it and reassemble it – in a uniform and reliable manner).

What can be avoided, however, is the necessity to entirely rewrite files – a time-consuming and resource-intensive operation with possible dangerous consequences – when adding metadata. If you are not fully convinced that metadata embedding could be a risky business, check out this post on the Lightroom Journal describing a potential data corrupting metadata bug in current Adobe flagships).

The first step towards a possible solution would be to get rid of hard-coded file offsets: they would disappear entirely and various parts of the files would be referenced by a logical name, for example “/Image/Thumbnail”, “/Image/Preview”, or “/Image/RawData”, and it would absolutely not matter where the actual elements are physically laid out in the file.

A directory, sporting a well-known name, would list the file’s content and readers could simply reference or access the various parts by name. One such part would of course be called something like “/Image/Metadata/EXIF” and could very welll contain said metadata in a format similar to the one currently used, namely a binary EXIF segment, containing XMP data and the Maker Notes segment as is does today.

The Maker Notes and EXIF data would only require minimal changes, namely the use logical names instead of file offsets where they store the location of the various file components. Existing metadata parsing code could be used with little modifications as the metadata itself would be stored in essentially the same format as it is today, save for the file offsets replaced by names.

The container itself would be a “filesystem-within-a-file” and use separate data streams for the various file constituents, image data – one stream per image representation – and metadata.

One huge advantage of such a container is that it would be possible to add, extend, truncate or remove data streams from the file without disturbing the other data streams it contains and, as such, extending the metadata stream within a file would simply be a matter of appending data to it or just rewrite the metadata stream entirely with complete disregard of the rest of the file: the storage container would take care of allocating the extra space needed, exactly like a file system like FAT, EXT3 or NTFS does for normal files on hard disks or memory cards.

Metadata updates would be much faster than when a full file rewrite is necessary and also perfectly safe as the physical layout of the various file constituents would be irrelevant, so an arbitrary amount of metadata could be added or removed without fear. Entirely new types of data or metadata could be invented and added in their own streams, and co-exist peacefully with existing data without breaking existing readers.

Since the actual binary or XML data would be stored similarly as it is today, changes to firmware and application software would be confined to the file I/O portion of the code, where the software would open the “/Image/Thumbnail” stream, for example, instead of seeking to absolute file location X, where the thumbnail is supposed to be stored.

The remaining of the file handling code could be essentially identical as the binary data (RGB pixels, sensor data…) retrieved from the various parts of the file would be exactly the same. Also, since the creation of those files would require an updated firmware, camera manufacturers could jump on the opportunity to add a new “Model ID” field in the form of a globally unique identifier (GUID), stored straight in the file’s header, that unambiguously identify the camera model so raw decoders and applications could identify the files beyond any doubt.

To sum it up, this proposed change is not much about data itself but mainly about how it is stored, more precisely in which type of container.

One readily available and widespread example of a suitable container would be the Microsoft Compound File, whose binary specifications are open and readily available, and which can be implemented on any platform or device, from camera phones to DSLRs to Linux and the Apple Mac, with little effort (all copies of the Windows operating system ship with an implementation since about 1995 and this format is already well known to the open source community). For those who’d see an issue related to the company that created this specification, think twice: Microsoft invented TIFF (with Aldus) and this doee not seem to be a problem to anyone…

With such a container it would be child’s play to define and add new types of streams, say “/Audio/MP3” or “/Video/H.264”, to existing files. Multiple formats could be stored side-by-side in the same file, such as “Audio/MP3″ and “/Audio/PCM”, if it makes sense for some application.

A software application could also add or rewrite, say, the preview image of any file without fear of breaking anything, for example in a stream called “/Image/Preview/JPEG”, which would contain a standard JPEG image (SOI, APP0, …, EOI).

Applications could actually look for such a preview stream and discover it at runtime without knowing anything about the format’s specifics. Likewise, an “/Image/Metadata/EXIF” stream could be found, read and updated by applications, again without the need to take care of - or even know – the details of the file layout. It would also be possible to add metadata to a particular image within the file without affecting other images in the same file, say “/Image/Preview/JPEG/Metadata/XMP”, if it makes sense, etc.

With a little coordination regarding the stream names and hierarchy, such a multimedia file format could become universal and used to store virtually any type of data with very little effort and maximum reuse of existing code bases and libraries. In-place extensibility, data discoverability and reliable updates would be key advantages compared to today’s container formats.

– Axel Rietschin (May 21, 2011)

What is the processor cache and how does it affect performance?

Image data is stored in main memory (RAM). The processor accesses the RAM through the memory bus, a communication channel that allows the data to move back and forth between the processor and the memory itself. Typically, when operating on pixel data, an image processing algorithm – say a sharpening filter – needs to grab some pixel’s values, perform some arithmetic operation on the values and store the resulting pixel back to memory. This process is then repeated with the next pixel and so on, until the entire image – or the entire selection – is processed. Worse, the algorithm often needs neighboring pixels (say the ones above and below the current pixel, as well as those to the left and to the right, plus the diagonal pixels) in order to compute the new value of a given pixels, causing the number of memory accessed to literally explode: that’s 8 extra read accesses for a small 3×3 convolution kernel raising the total number of memory reads to process a 12MP image to about 340 million memory reads to access all 3 color planes, or close to 1 billion memory reads for a 5×5 kernel running over the entire image.

The problem is that the memory bus and the memory itself do not operate as fast as the central processor. For example the processor clock may run at a speed of 3GHz but the memory may only run at 1333Mhz, or 1.3GHz, which is considerably slower. The processor essentially waits until the memory responds and transfers the data, thus, by some stretch of imagination, one can think the computer operates closer to the 1.3GHz memory speed than to its 3GHz processor speed when is accessing memory very often, as when running image processing filters.

To mitigate this performance bottleneck, processor manufacturers added a small amount of very fast memory close to the processor itself. When accessing main memory the cache is filled with data under the assumption that it will be needed again, and on subsequent accesses the data is returned from the very fast cache if it’s still there, which spares the processor to reaching out to main memory and waiting for it to react and return the data.

A complete description of CPU caches is beyond the scope of this article but it boils down to the following: the larger the cache the more data it can keep ready for fast access and the less slow down there is when accessing memory. When the data is not is the cache – either it was not accessed yet, or it was accessed a relatively long time ago and was discarded since then, a “cache miss” situation occurs and the main memory is accessed to retrieve the data.

The above is only a tiny part of the story as the cache also stores memory writes and the whole cached memory system is actually quite complex, in particular in the presence of multiple processors with separate caches that must be kept coherent etc, but again, we don’t need to dig too deep in the details: what we need to take with us is “the more cache, the better”.

Pentium 4 processors came with 128KB of cache, then 256KB and finally 512KB. Contemporary processors like the Intel i7 can have up to 12MB of cache, or 24 to 96 times more than the old P4’s. Needless to say, just because of that, an i7 will outperform a P4 by a considerable margin (huge, in fact) even if clocked at the same processor speed, and in cache misses situations the current RAM chips, running at 1333, 1600, 1800 or 2000MHz will supply the data considerably faster than before, adding to the overall performance.

Image processing is a bit special special in the sense that the algorithms needs to access data all over the place (for example when accessing pixels above or below the current one, the memory locations needing to be reached are quite far apart from each other), in technical terms one say they exhibit poor locality of reference. The consequence is that this type of behavior quickly overwhelms small memory caches and makes them much less effective. With relatively large cache sizes (8MB or more) a significant portion of the image can be kept right in the processor cache, dramatically speeding up access to pixel data. To keep things in perspective, server processors such as the Intel Itanium have a relatively low clock speed (about 2GHz) but a gigantic 24MB CPU cache, making them ideally suited for intensive number crunching over large data sets.

We are now ready for a quote from Intel on the subject “In recent years, the resolution of image sensors has increased significantly, but image processing algorithms have not improved in efficiency. The bottleneck for nearly all image processing algorithms is memory access, and access time increases significantly when image data resides outside the L2 cache. Even with more computing power available, the increased size of the images impacts the ability to achieve high performance due to the increased number of cache misses”.

Wrap up and conclusion

Memory accesses represent a significant bottleneck for image processing and a large CPU cache helps mitigate the issue. If given a choice, always pick the processor with the largest CPU cache, all other things being equal it will perform better for image processing than those with a smaller cache, even if the latters are clocked faster! Of course, you also want multiple cores, say 4 or more, and a modern 64-bit operating system with plenty of memory (12GB seems to be enough at the time of this writing) to gain the ability to smoothly multitask multiple memory hungry applications such as FastPictureViewer, Adobe Lightroom, Nikon Capture NX and Adobe Photoshop all together, with enough room to run your favorite productivity applications like email and instant messaging on the side.

RELATED: The More Cores, the Better.

The product takes advantage of the open architecture of Microsoft Windows to provide enhanced image file formats support to the operating system. Once the image decoders – also known as “codecs” – are installed on a given computer, Windows Explorer is able to open and display new image formats in a natural manner, similar to the way it handles standard image formats such as JPEG and TIFF, complete with EXIF data display, Windows Photo Viewer slideshows and desktop search integration.

The software also supports specialized image formats like OpenEXR, Radiance HDR, Truevision Targa and DirectX DDS, as used in the computer gaming and graphics industry.

“Codecs are difficult to author and traditionally troublesome components” said Axel Rietschin, founder of the company. “We took extra-extra care and paid minute attention to details to make sure our product is dependable”. “Installing the Codec Pack is like lighting up Windows Explorer” said a local photographer, early user of the product. “Sorting Canon RAW images on my new 64-bit laptop is now so much easier”.

The FastPictureViewer Codec Pack was designed from the start to work with Microsoft Windows 7 and is listed in the official Windows 7 Compatibility Center. Windows Vista is also fully supported as well as Windows XP, with some restrictions. All current 32-bit and 64-bit editions of Windows are supported.

More information and trial downloads available on the web at

www.fastpictureviewer.com/codecs

About Axel Rietschin Software Developments

Axel Rietschin Software Developments is a niche innovator of imaging software and components. Founded in 2008 and headquartered in Geneva, Switzerland, the company specializes in developing software solutions that meet business and consumer needs. Also known for its FastPictureViewer Professional photographer-oriented image viewer, the company’s technologies have been used by individuals and organizations across the world.

Axel Rietschin Software Developments
36 rue de Carouge
CH-1205 Geneva
Switzerland
www.fastpictureviewer.com

Contact Person:

Axel Rietschin, founder
web{@}fastpictureviewer.com

# # #

Future software may make better use of multiple cores in more places, such as when processing a single image, by increasing the level of parallelism in the program code and reducing the granularity of the parallel regions: today the level of parallelism is very coarse (file level, as in processing multiple files in parallel during batch export operations). Tomorrow a histogram or sharpening or color conversion filter will be computed in parallel on multiple cores while processing a single image, and those operations will take a fraction of the time they takes today, with a dramatic positive impact on application responsiveness.

In fact, some applications already try to optimize CPU usage in specific places, for example the Save-for-Web function of our own FastPictureViewer Pro’s batch file processor processes multiple files in parallel when the program is running on multicore computers. Also, when processing individual files, the optional sharpening filter than can be applied to exported images is itself computed in parallel on up to three CPU cores: one for each of the Red, Green and Blue color channels. Additionally, the batch processor takes special cares to avoid trashing the disks by throttling its data throughput and, while not necessarily topping the CPU usage charts all the time, exhibit a much higher overall throughput that what could be achieved in a standard way where each image is processed in turn. The throughput is also higher than what would be achieved by simply processing as many files in parallel as they are cores in the system, with each file being processed sequentially on a single core.

Application developers need to revisit all the time-consuming portions of their program code and identify the candidates for parallelism. The algorithms then needs to be rewritten in a parallel fashion and sometimes the program’s flow needs to be restructured in a number of ‘tasks’ that can run independently. By doing so, developers increase the level of latent parallelism in the program code, and when said programs are executed on multi core hardware, actual parallelism takes place and the processing time decrease accordingly, increasing the application performances.

Clearly, those changes will not happen overnight. Multithreaded programming, where an application literally executes different parts of itself concurrently, is a very difficult art and fine-grained parallelism is even harder as it requires a complete rethinking of every minute detail of the program’s operations. Finding errors in parallel program code is a whole new challenge in itself and takes the art of debugging to an entirely new level, so it is likely that the shift to truly multicore–enabled parallel applications is going to be made incrementally with small steps at every release, in particular in large existing applications with a huge code base and many legacies.

We are only at the beginning of the multicore/manycore era and much of the benefits are still to come, in the form of software upgrades from our favorite software publishers.

The number one thing to avoid is to download and run any 3rd-party utility that supposedly “optimizes” or “clean” stuff on your computer. Those tools don’t do anything useful at best (beyond what Windows already offers) and in the worst cases they may wreak havoc with your machine beyond all hopes of repair.

Some of these utilities gets warm recommendations from random (and anonymous) internet users presenting themselves as “experts” in tech and not-so-tech forums, but let me give you some insight from the other side of the fence: software I have written or helped write in the past two decades collectively runs on tens of millions of computers. FastPictureViewer alone is closing fast on its first million installs and I can tell you that almost every time a user reports an installation issue (such as inability to install, uninstall or upgrade something) the problem could be traced back to either overzealous “cleanups” that removed vital registry entries or files from the computer, or alterations to the default security on registry hives that sometimes removed all rights to the Administrators or even the SYSTEM account itself (supposedly to prevent malware installation), creating inextricable situations where no one has the rights to restore the proper access rights…

The issues caused by those utilities can be insidious: everything appears to work as before (remember: they don’t do anything useful) but a problem might crop up months later when a round of automatic updates fails to install properly, leaving the computer in a broken state, or a service pack needs to be installed. Those adventurous users playing with dangerous things such as beta service pack might find themselves in the uncomfortable situation where something that cannot be uninstalled prevents something else from getting installed, creating a deadlock whose only issue in to reinstall the system from scratch.

Of course, when this happens, the user always points his finger to the innocent software publisher (or operating system vendor) and never to the freebee that silently messed up the computer in the first place. In particular, never, ever run anything that pretend to speed up your computer by “cleaning” your registry. There are millions of values in the registry and the access time to a particular one is largely independent of the total number of keys and values. Removing a few hundred “unused” keys (or even thousands) is not going to have any impact whatsoever on your computer performance. Nothing, nada, rien, niet, niente, niets, zero - just don’t do it.

Okay, but now what?

Run Disk Cleanup

Clean temporary and cached files from time to time: Windows has a built-in Disk Cleanup utility that works just fine for that. Click the Start button and type Disk Cleanup in the search box (XP users can simply run cleanmgr). This utility takes care of temporary files left behind, empties the Recycle Bin and removes unused stuff without messing up with anything vital. Disk Cleanup can be started from the command line and can be scheduled to run automatically on Windows XP or later operating systems. The Disk Cleanup tool will clean the Windows Thumbnail Cache by default, which may or may not be desired. If you want to keep the cached thumbnails, just uncheck the “Thumbnails” option in the list of stuff to clean up.

Defragment Your Hard Disk Drives

The second (and last) thing that you need to do to keep your computer in shape is to defragment your drives regularly. Again, Windows has all it takes to perform this task, and recent versions of the operating system (Windows 7…) actually became very smart about defragmentation and fast bootup. To make a long story short, the operating system watches what you do (what gets loaded during the boot sequences, what program are started and how often) and optimizes all it can: it gets smarter as you use it by trying to predict what will be needed next (prefetch), and optimize disk file placements according to the load sequence during boot. Learn how to defragment your hard drives, then schedule disk defragmentation to run automatically (XP, Vista and Windows 7) and forget about it.

Advanced users can force a full profile-based boot sequence optimization using the xbootmgr program, part of the Windows Performance Analysis toolkit. While not recommended for normal users, the Windows Performance Toolkit (WPT) contains a utility that profile and trace the computer’s boot sequence and optimize file placement in consequence. Get the toolkit and run the following command:

xbootmgr -trace boot -prepSystem -verboseReadyBoot

…from an administrative command prompt (your computer may reboot up to 6 times for the tool to learn and discover the optimal settings, so this process will take some time).

There is not much to do, beyond scheduling Disk Cleanup and the Disk Defragmenter, to keep your computer in normal shape. Leave page file management to Windows (unless you really know what you are doing), uninstall applications that you don’t use to free up some space and opt for Microsoft Security Essentials if you have a choice: it’s lean and mean and does not bog down your computer as much as some other AVs, free and commercial alike.