Post processing

Right from the outset, 12 years ago, post processing became an integral part of my photographic experience. I haven't counted, but I expect I have spent much more time processing images than I have spent capturing them.

I enjoy post processing, learning how to do it better, experimenting with different techniques and products, discussing it with others. I can't imagine simply using images straight out of the camera. 

I am convinced that post processing can make a big difference to the look of images and can usually (for my images, and my visual preferences) improve them compared to what comes out of the camera.

I hope that my post processing has improved over time. One of the things I did while preparing these posts was to reprocess some images with my current processing products and workflow to see how they compare to how I processed them four to five years ago in 2014. Two were captured with a Panasonic FZ200 small sensor bridge camera, two with a micro four thirds Panasonic G3 and two with an APS-C Canon 70D. All six are small aperture shots with image quality dominated by the softening effects of diffraction (the first five used apertures equivalent to around f/45 on full frame, the last one an aperture equivalent to around f/26 on full frame). I believe diffraction-softened images like this are particularly susceptible to improvement in perceieved sharpness/detail from effective post processing.

Click on an image to see a larger version

Comparison 1: Panasonic FZ200, f/8, 1/250 sec, ISO 100, flash

Comparison 2, Panasonic FZ200, f/8, 1/640 sec, ISO 100, flash

Comparison 3: Panasonic G5, f/22, 1/160 sec, ISO 160, flash

Comparison 4, Panasonic G5, f/22, 1/160 sec, ISO 160, flash

Comparison 5, Canon 70D, f/32, 1/250 sec, ISO 800, flash

Comparison 6, Canon 70D, f/16, 1/80 sec, ISO 1600, available light
(I made a mistake in processing this one and included the extra sharpening in Silkypix that I normally use only for invertebrates. The fine crystalline structure of the petals is therefore probably a little overdone.)

I know, because people have told me, that my post processing looks to be complicated and time-consuming. That is not how I see it. I deal with large numbers of images and over the years I have deveoped a workflow that lets me deal with these large sets of images quite quickly and with much less effort than might appear from a description of the process. This involves using several pieces of software, using each for what it is particularly good at, and using batch processing to get images into a state where they are easy to handle before I start making any adjustments to individual images.

I currently use between two and six software products while processing a set of images. Depending on the circumstances this may include some or all of the following:

• Fast Picture Viewer, for an initial, very rapid trawl of the images to get rid of the obviously unusable ones. Fast Picture Viewer uses the small JPEG images embedded in raw files rather the raw files themselves as used by Fast Raw Viewer, but for my purposes the embedded JPEGs are good enough and Fast Picture Viewer has a macro facility which I use to file the selected (actually, the non-rejected) images by ISO, which makes DXO processing much easier.
• Either DXO PhotoLab (for stills) or Helicon Focus (for stacks) for the first stage of the processing, using PhotoLab for noise reduction of noisy small sensor raw files and using Helicon Focus for botanical subject stacking and associated retouching to cure or at least hide stacking problems.
• Sometimes, Silkypix to add extra sharpening/revealing of fine detail for invertebrates and/or rescuing colours from very bright areas, especially for flowers.
• Always, Lightroom, to apply image-specific adjustments such as cropping, adjusting tonality or localised sharpening or noise reduction, and select the images I want to keep.
• Very occasionally, Photoshop to do some additional processing that I can't do in Lightroom, for example complicated cloning.
• Quite often, XNView to review the processed images as output by Lightroom and see if I still want to keep them or not, or see if they need some additional processing.
• Always, Faststone Image Viewer to deal with the bureaucracy of renaming images, deciding what order to display them in and make backup copies locally and on line.

I use batch processing when using PhotoLab and/or Silkypix. In the case of PhotoLab the presets I use are specific to a camera and ISO. The PhotoLab batch processing can be very time-consuming, but it only takes a very small amount of time to set it up and the rest of the time I can leave it to cook and do other things.

Sensor size and image quality

This is yet another topic that generates intense debate. All I'm going to do here is to say what conclusions I have come to from a practical perspective for my types of subject matter and my working methods.

I like having a camera which is small enough to put into a pocket but which can cover a very large range of focal lengths from wide to telephoto. For the range I want it seems inevitable to me that the camera must use a very small sensor and one must simply put up with the consequences for image quality. I don't think this is at all controversial.

For invertebrates, where I used minimum aperture almost all the time, I don't think it matters what the sensor size is because the effects of diffraction limit image quality to a similar level for all sensor sizes. I have a lot of experience and examples with sensors of 1/2.3", micro four thirds and APS-C size. I have not had a chance to test this yet with my full frame camera. I will be very surprised if that is any different from the others in this respect, but as always I would prefer to be evidence-based on this rather than just leaving it as a supposition and so I intend to run some tests in due course.

For flowers and other botanical subjects I have produced images that I like using cameras with small sensors, but I tend to prefer the look of botanical images captured with sensors of micro four thirds size or larger. For single capture images I have not developed any preference as between micro four thirds and APS-C, having used both quite a lot for botanical subjects. In two exercises where I compared images of the same subject captured with a micro four thirds camera and a full frame camera, here, and here, I could not discern any image quality benefit of using full frame. For stacked images I can only use my Panasonic cameras and I have a strong preference for the micro four thirds G9 over the G80 and the small sensor FZ330.

I am part way through comparing micro four thirds and full frame for sunsets and cloudscapes. Early indications are that full frame may produce noticeably better image quality for sunsets, both for high dynamic range scenes where the sun is still above the horizon and for low light scenes after the sun has set. That is for single shots. For exposure bracketed shots combined using exposure fusion ("HDR") there may be little difference for scenes with high dynamic range when the sun is still above the horizon. However, more thorough testing may change these early impressions.

RAW or JPEG?

This is another question that is the subject of periodic, and sometimes ill-tempered, debates, especially at dpreview.com. My own view is that which is best depends on the context. (And sometimes it may be better to use both and decide afterwards which to use, although as it happens that is not an approach I have ever used much.) 

I started out using JPEG because at first I didn't know about raw. Then I knew about it but my cameras did not. Then my cameras knew about it but I didn't know how to deal with raw files. Then I knew about how to deal with raw files (at least in outline) but wasn't convinced it was worth the extra effort. Eventually I came to the conclusion that there were advantages to using raw and as I used raw more I came to realise that it didn't involve any more effort than using JPEG, and in some cases actually less effort.

It remained like that for some time, until I started using stacking. For focus bracketing with the Panasonic G80 I quickly switched from using raw to using JPEG because with raw the buffer would fill up and then the capture rate would become extremely slow. But I didn't use focus bracketing much because I found that I preferred using post focus to capture videos from which to extract JPEG images to stack. So now I use video-derived JPEGs most of the time for botanical images, and raw for everything else.

I took an active part in a very long (and in this case good-natured) series of threads at dpreview about JPEG and raw, discussions that I found very informative. (This post at dpreview contains links to the 12 threads concerned.)

Aperture and exposure bracketing

I have occasionally used exposure bracketing, sometimes for sunset scenes with the sun in the shot or, on only a handful of occasions, for botanical shots with a flower against the sky. Mostly though for contre-jour botanical shots I expose for the highlights and do deep shadow lifting. On reflection, it might be better if I used exposure bracketing more often for both these types of shots.

On the other hand I have used aperture bracketing a lot, always for botanical scenes. For me, one of the issues with botanical shots, especially flowers, is trying to get a pleasing balance between the amount of the subject that is in focus and the look of the background, both of which change as the aperture changes. Before capturing a shot I can't tell how that balance is going to work out for a particular aperture. Often, with micro four thirds or APS-C, it may be around f/8 to f/13, but from time to time it can go as far as f/4 or f/16, and occasionally right out to f/22 or, very, very rarely f/2.8. Trying different apertures and reviewing them on the LCD slows down the flow of a session, giving it a very different feel, dulling my creativity. And in any case it can be ineffective because it is difficult to judge accurately from what appears on the LCD, especially in bright sunlight, and even more so if the capture has been underexposed to protect highlights.

Starting with the G80 and now with the G9 I can use aperture bracketing; with one press of the shutter button I can have exposures all the way from f/2.8 to f/22, and I can then take my time choosing which to use when viewing them in subdued light on my PC, if necessary after some initial processing to lift the shadows. I have found this to be very effective for my purposes, in terms getting results that I like the look of and also in terms of keeping capture sessions moving at a good pace.

There is a downside to aperture bracketing that needs to be weighed against the advantages. The ISO is constant during an aperture bracket sequence. That means that the shutter speed doubles in length for each stop reduction in aperture. I therefore need to use an ISO that will give a tolerable shutter speed at the smallest aperture that I am likely to want to use. Quite often that means using ISO 800 or sometimes ISO 1600, and even then the f/16 and f/22 exposures may sometimes be too slow to avoid motion blur. That means that if I use one of the shots with a more typical aperture then the ISO will be higher than I would have set it if I had been shooting a single shot with that aperture. So, using aperture bracketing, ISO is suboptimal a lot of the time, which can make the processing more complicated than it would have been for a lower ISO single shot, and my degrade the perceived image quality in some cases.

On balance I thought the advantages of aperture bracketing outweighed the disadvantages, and so I used it a lot with the G80 for botanical subjects, at first mixing it with single shots and then eventually using it most of the time without using single shots.

This changed when I got the G9 and soon started using focus stacking most of the time for botanical subjects.

I have never tried aperture bracketing for invertebrates because getting the maximum DoF coverage of the subject is my primary concern for invertebrates. 

Single shots/stacks

I have recently been using stacking most of the time rather than single shots for botanical subjects. I had a slow start from late 2016 when I first tried stacking using post focus with a Panasonic FZ330. This was limited to experimental use of stacking, as was my first year or so with a Panasonic G80. 

The reason my use of stacking was so limited was that I encountered sufficient problems with doing the stacking that it put me off. The main problem was halos, but there were also issues with ghost images and backgrounds being very noisy and/or posterised and looking very ugly. Dealing with images with those problems could be very time-consuming, more so than seemed worthwhile. It was only during 2018 that I slowly came to a better understanding of the causes of the problems and how to deal with them, and when to give up because the problem was too difficult to be worth trying to solve.

With the FZ330 I was limited to using post focus to derive JPEGs for stacking, and this used 4K video from which I could extract 8 megapixel JPEGs. The G80 also did post focus, with 8 megapixel 4K video, but it also did focus bracketing. This allowed the use of full size images, raw as well as JPEG. It therefore seemed the better bet in terms of image quality. 

Focus bracketing could indeed produce very good results, but there were downsides. I like to work hand-held, and I could do this when using post focus, which captures frames at a rate of 30 frames per second. This meant that the time needed to capture the images was typically from between two and six seconds, which worked fine hand-held. Focus bracketing worked over six times slower, at around 4.5 captures per second. That made capture times uncomfortably long for hand-holding. Also, managing large numbers of individual images, especially large numbers of raw files, proved significantly more onerous than managing a single video file per capture. 

I was also not convinced that using raw rather than JPEG or using focus bracketing rather than post focus produced better image quality. That seemed to be determined largely by how well the stacking worked and what measures I had to take to cure the problems.

It was when I got the G9 in late 2018 that my use of stacking really took off. The G9 can use 6K post focus and this produces 18 megapixel images, which is actually more pixels than full size G80 images. Also it lets you use a very flat camera profile, Cinelike D, and I found that using this coupled with minimising JPEG noise reduction and sharpening led to stacked images from Helicon Focus with tonality that I liked and which responded well and easily to post processing, although much to my surprise, they turned out to need very little adjustment a lot of the time.

And what about the halos, ghost images and nasty backgrounds? They seem to be less of a problem now. Partly this may be because Helicon Focus has had an upgrade and can deal with some of the issues better.  And partly it may be a matter of having more experience with the stacking software and coming to a better understanding of how to use it effectively, and how to use other software such as Lightroom and occasionally Photoshop better to deal with problems that Helicon Focus can't. And perhaps I am a bit more relaxed about the imperfections. I have come to realise the obvious reality that single shot botanical images have their own issues and that on balance even with their imperfections I often prefer stacked images to single shot images of the same subjects with their different imperfections.

I think my main problem with botanical stacking at the moment may be that I am using it too much, hardly using single shots at all. Looking back through a large number of botanical images in preparing these blog entries I realise that there are many that I like just as much as I like some of the recent stacked images, and so I would probably do best to use both single and stacked imaging for botanical subjects. The G9 makes it quick and easy to switch between exposure bracketing and post focus.

As for invertebrates, I know that it is possible to produce wonderful stacked images of the sort of subject matter and magnifications I deal with using kit that I have - G80 or G9 with and without Raynox lenses and/or extension tubes. I am hugely impressed with what this photographer does with stacked images of invertebrates using this kit, and using it hand-held I think. However, in my limited experiments so far with a couple of dead insects I have found around the house my results have not been good. I will try with live subjects when the invertebrates reappear, but it may be this is a tecnique that is not going to work for me. One thing that has become clear in discussing techniques and results with other close-up/macro photographers over the years is that what works for one person may not work for another, even amongst equally competent photographers. 

Auto/manual focus

I think the consensus view is that autofocus is not much use for close-up/macro. My experience is different, and I use autofocus almost all the time. It does however depend on the kit and the magnification. 

For my relatively low magnification botanical imaging I find autofocus fine with close-up lenses, not quite as good but very usable with the Olympus 60mm macro and not quite as good again but usable with my Sigma 105 macro on APS-C and full frame. This is for single image capture. The situation is different for post focus and focus bracketing, both of which use autofocus. The 60mm macro works fine with post focus and focus bracketing (neither post focus nor focus bracketing is available on my APS-C and full frame cameras).

For invertebrates I find autofocus with close-up lenses fine down to 10mm scene width or so. It works less well as the magnification increases beyond that but it can still be usable for quite small scenes. I have used autofocus (hand-held) with a scene around 3mm wide (as can be seen from 13:35 to 14:00 in this video at You Tube where I was using a 32 diopter Raynox MSN-505 on a 45-175 lens on a Panasonic G5, which at maximum zoom covers a scene width of around 3mm). That was exceptional though. I have only ever worked with scene sizes that small on an occasional experimental basis.

My experience of autofocus with macro lenses has not been positive for the scene widths I normally work with for invertebrates. I found that the focusing hunted so much that trying to use autofocus was unproductive.

And of course higher magnification macro lenses do not support autofocus anyway.

Why such small apertures?

For invertebrates I use very small apertures. In fact I use minimum apertures. Most people won't use apertures that small because diffraction causes loss of sharpness and detail. However, there is a trade-off. If you use larger apertures you can get better fine detail, but in a thinner front to back slice of stuff that is in focus. With very small apertures you lose that fine detail, but you get more larger scale detail across a larger front to back slice. For my taste, and it seems for quite a lot of people who see my invertebrate photos, that works well. As for the lost detail, if you are looking at a single image rather than comparing two taken with different apertures you may not even realise there is detail missing. Also, some (or sometimes even all perhaps) of the fine detail that is missing may be too small to see anyway, depending on the size of the displayed image, how close you get to it, and whether you can zoom in to see greater detail.

The following illustration is not very pretty, but it is difficult to arrange this sort of comparison with a live subject. The illustration flips every three seconds between f/5.6 and f/22. The f/5.6 version should be sharper/more detailed because f/5.6 is somewhere around the sharpest "sweet spot" aperture for that lens, whilst diffraction has a big impact at f/22. However, when I pixel peeped at 1:1, concentrating on the small areas that were in best focus in the f/5.6 version, I wasn't sure they were sharper or more detailed than the same areas in the f/22 version. This may be because the centre of focus shifted slightly between shots and confused the issue. Also, the processing may have had a stronger effect on the less sharp f/22 version. In any case, my eye is not upset by a loss of fine detail in the f/22 version. On the other hand the difference in how much is in focus seems large to me, and I very much prefer the f/22 version with it having so much more in focus, and that is why I use such small apertures for invertebrates.

The situation is different for botanical subjects and for common birds in flight.

What aperture I use for botanical subjects depends on whether I am using stacking or not. If I am using post focus to capture videos for stacking then I use f/2.8, which is maximum aperture on the 60mm macro. This is not the best aperture for sharpness; that is around f/4 to f/5.6 depending on whether you are more concerned with centre or edge sharpness according to the resolution chart here at ephotozine.com.  However, I often work hand-held, sometimes getting close to, or hitting, the 1/30 sec lower limit for post focus, and using f/2.8 keeps the shutter speed faster than it would be with a sharper aperture. I find f/2.8 sharp enough for my purposes and as with the loss of sharpness/detail from diffraction, what I am losing would quite likely not be visible at my output sizes anyway.

Of course, the light level is often high enough to be able to get a fast enough shutter speed with a smaller aperture, so I could use the sharper f/4 to f/5.6. However, there is quite a lot to think about for each capture and leaving the aperture at f/2.8 simplifies things a little and I am content to leave it there as one less thing to have to deal with. This is suboptimal in terms of potential image quality, but practical out in the field, and after a certain point with image quality, for me, practicality becomes the most important factor.

And there is another factor too. Using f/2.8 maximises the out of focus rendition of the background. That is good for separating the stacked subject from the unstacked background, but bad in that it is more likely to produce ugly, too fast transitions between in focus and out of focus areas. Also, as desribed below for aperture bracketing, maximising the out of focus rendition of the background may not be optimal (to my eye) for the overall visual appeal of an image. However, these considerations are too difficult (for me) to judge before I see a stack developed on my PC, and I am content enough with what I get from f/2.8 to take what I get from this unoptimised approach of always using f/2.8.

The trouble, for me at least, with optimising each capture in every respect is that it requires a slow, careful, measured, thorough, deliberative approach which can soak up a lot of valuable time out in the field, decreasing the number of subjects/scenes tackled during a session. For me this is a significant consideration in what are often target rich environments. 

Also, full spectrum optimisation can increase the opportunity for error. For example, when using post focus for stacking (which involves using JPEGs with "baked in" white balance) I think I get best results by setting the white balance individually for each scene from a grey card. The trouble is that I risk using the wrong white balance, possibly badly wrong, if I move from say a shady scene to a sunny one and forget to reset the white balance (I am working to develop a strong habit of always setting the white balance for each scene as the first step). This approach involves taking the time to do what are redundant white balance settings when I move from one scene to the next with similar illumination, but the white balance is important enough to me to take that hit (which is not serious as I have set up the G9 so setting the white balance from a grey card is quick to do) and to take the risk of sometimes using a badly wrong white balance.

If I am not stacking a botanical scene then the choice of aperture is solely down to the overall visual appeal of the image, especially the balance between the amount of the subject that is in focus (smaller apertures often better) and the way the background in rendered (larger apertures often better). For single images I can optimise the shutter speed and ISO for a particular apertue. However, I generally use aperture bracketing as this is much faster in execution, but this means taking whatever I get in terms of noise. This is because the ISO has to be chosen so as to give the smallest aperture shot a fast enough exposure, and this means that the ISO is almost always higher than would be the case for an individually optimised exposure.

On looking back through some captures that used aperture bracketing it turns out that I rarely use the wide open f/2.8 captures. This is mainly because, usually, not enough of the subject is in focus at f/2.8. Also, I don't always use the capture that (to my eye) just has enough of the subject in focus and which throws the background out of focus more and isolates the subject more than a smaller aperture would. I sometimes prefer versions where the background is less out of focus.

For common birds in flight I use a particular aperture (f/13 on the Canon 70D) and also a particular ISO (ISO 800) as these settings give me a good combination of depth of field and fast shutter speeds on the bright days I do this type of photography.