LUT for viewing CF raw

After some messing around I finally made a LUT that makes raw cineform files look like both transforms discussed in previous post (sensor > XYZ and XYZ > sRGB) have been applied.

EDIT: lut file link is broken at the moment, try to fix it soon...
Download the LUT file (.cube format) here:

Register the LUT file by doubleclicking on it (Windows), this installs it to Firstlight LUT directory.

To apply the lut, make sure that input and output curves are set to the same setting and that primary corrections are disabled. For some reason enabling primary corrections makes image slightly pink in highlights.

This lut works nicely in NLEs and video players but for some reason Resolve does not show any Cineform corrections. It seems that Resolve is able to reference raw file directly and bypasses Cineform renderer (confirmed in bmcuser forum by David Newman from Cineform)

WB correction, dammit!
Previous LUT makes image viewable but it still looks a bit too green. This is due to missing white balancing step which I tried to get working and got working (kind of) but I'm not sure I made it correctly.

DNG files contain metadata called as-shot-neutral which should be the value which is considered white (according to which illuminant? must dig more). In BMCC files, values read something like 0.63 1.00 0.79. If we take them as RGB values, it doesn't make any sense because RGB white should be 1.00 1.00 1.00. I figured it must be in either sensor values or in XYZ. Messing around with it, I thought that applying sensor > XYZ transform to this colour would give me white in XYZ space that I could then use to grade raw data in XYZ space by setting it's white point to my values. It works, kind of, because it removes the green tint but colors still seem a bit off.

Some sources (dcRaw developer for example) suggest that WB correction values to be applied before debayering because it prevents some debayer artifacts. If I could do it, I would try it... Seems that I must code my own test engine to try out my genius ideas... maybe even make it work on CUDA...

BMCC RAW to Cineform RAW

EDIT: useful links
http://lclevy.free.fr/cr2/
http://www.guillermoluijk.com/tutorial/dcraw/index_en.htm

One of the hot topics in BMCuser forum is the Cineform Conversion and particularly the raw dng to Cineform raw possibility. As Cineform can convert dpx files to cineform raw files (with dpx2cf command line utility), this could be one way to deal with large file sizes that working with raw files tends to bring upon.

NOTE: Older Cineform versions don't seem to support dng conversion, you need release from at least april/may of 2012! Otherwise you get the "DPX file not found" error.

DNG sequence to Cineform RAW file

Converting dng sequence to cineform is pretty easy. Use dpx2cf tool with, for example, following command:

dpx2cf.exe Frame*.dng shot1_cf.avi -q5

This reads all dng files from shown source (* marks frame numbers) and converts it into cineform file. Additional settings like frame rate etc can be set, see settings by simply running the program in command prompt.

EDIT: there was a typo in command example, there should not have been a " mark after shot1_cf.avi.

Cineform RAW looks green?

After conversion it is immediately apparent that something is wrong. Image looks very green and strange. This is because Cinefrom converter does not apply any correction on the image and we see pure raw (debayered ofcourse) image. As sensors are usually more sensitive to green light, image looks green.

Raw processing software (like Lightroom or Camera Raw) applies camera calibration correction before showing the image to user and so we don't really know what the sensor actually sees. Because dng and other raw formats contain these profiles in their metadata, raw conversion software usually knows how to perform this correction. Cineform converter on the other hand does not use this info, puts raw sensor values straight into the file and also shows us these values without correction.

Reading DNG metadata

To obtain metadata from dng file, we need some software that can extract that info from the file header. For example a handy set of tools called EXIFUtils. With this tool we can extract all the interesting metadata that BMCC dng files have to offer.

For example:

Camera
  image-type                    : Main
  main-width                    : 2432
  main-len                      : 1366
  main-bits-sam                 : 12
  main-comp                     : None
  photo-int                     : Color Filter Array (CFA)
  strip-off                     : 9856
  orient                        : Upper Left
  sample-pix                    : 1
  row-strip                     : 1366
  strip-cnt                     : 4983168
  planar-conf                   : Chunky
  cfa-rpt-pat-dim               : 2 2
  cfa-rpt-pattern               : 01000201
  tag-9216                      : 00000001
  dng-version                   : 01020000
  dng-model                     : Blackmagic Cinema Camera
  dng-lin-table                 : 54 55 55 56 57 58 59 59 60 61 62 63 63 64 65
                                  66 66 67 68 69 70 70 71 72 73 74 74 75 76 77
                                  78 78 79 80 81 81 82 83 84 85 85 86 87 88 89
                                  89 90 91 92 93 93 94 95 96 96 97 98 99 100
                                  100 101 102 103 104 104 105 106 107 107 108
                                  109 110 111 111 112 113 114 115 115 116 117
                                  118 119 119 120 121 122 122 123 124 125 126
                                  126 127 128 129 130 130 131 132 133 134 134
                                  135 136 137 137 138 139 140 141 141 142 143
                                  144 145 145 146 147 148 149 149 150 151 152
                                  152 153 154 155 156 156 157 158 159 160 160
                                  161 162 163 164 164 165 166 167 167 168 169
                                  170 171 171 172 173 174 175 175 176 177 178
                                  179 179 180 181 182 182 183 184 185 186 186
                                  187 188 189 190 190 191 192 193 194 194 195
                                  196 197 197 198 199 200 201 201 202 203 204
                                  205 205 206 207 208 209 209 210 211 212 212
                                  213 214 215 216 216 217 218 219 220 220 221
                                  222 223 223 224 225 226 227 227 228 229 230
                                  231 231 232 233 234 235 235 236 237 238 238
                                  239 240 241 242 242 243 244 245 246 246 247
                                  248 249 250 250 251 252 253 253 254 255 256
                                  257 257 258 259 260 261 261 262 263 264 265
                                  265 266 267 268 268 269 270 271 272 272 273
                                  274 275 276 276 277 278 279 280 280 281 282
                                  283 283 284 285 286 287 287 288 289 290 291
                                  291 292 293 294 295 295 296 297 298 298 299
                                  300 301 302 302 303 304 305 306 306 307 308
                                  309 310 310 311 312 313 313 314 315 316 317
                                  317 318 319 320 321 321 322 323 324 325 325
                                  326 327 328 328 329 330 331 332 332 333 334
                                  335 336 336 337 338 339 339 340 341 342 343
                                  343 344 345 346 347 347 348 349 350 351 351
                                  352 353 354 354 355 356 357 358 358 359 360
                                  361 362 362 363 364 365 366 366 367 368 369
                                  369 370 371 372 373 373 374 375 376 377 377
                                  378 379 380 381 381 382 383 384 384 385 386
                                  387 388 388 389 390 391 392 392 393 394 395
                                  396 396 397 398 399 399 400 401 402 403 403
                                  404 405 406 407 407 408 409 410 411 411 412
                                  413 414 414 415 416 417 418 418 419 420 421
                                  422 422 423 424 425 426 426 427 428 429 429
                                  430 431 432 433 433 434 435 436 437 437 438
                                  439 440 441 441 442 443 444 444 445 446 447
                                  448 448 449 450 451 452 452 453 454 455 455
                                  456 457 458 459 4
  dng-bl-lvl-dim                : 1 1
  dng-bl-level                  : 256
  dng-wh-level                  : 60074
  dng-def-crop-origin           : 16 8
  dng-def-crop-size             : 2400 1350
  dng-color-matrix-1            : 1.31 -0.50 0.01 -0.42 1.44 0.05 0.07 0.22 0.
                                  73
  dng-color-matrix-2            : 1.01 -0.27 -0.08 -0.49 1.34 0.11 -0.06 0.33
                                  0.54
  dng-camera-calib-1            : 1.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 1.00
  dng-camera-calib-2            : 1.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 1.00
  dng-as-shot-neutral           : 0.63 1.00 0.79
  dng-base-exp                  : 2.40
  dng-calib-illum-1             : 17
  dng-calib-illum-2             : 21
  tag-c763                      : 0859011400000000
  tag-c764                      : 24.00

We get a lot of info about file size, black levels, crops etc. And some color matrices.

DNG color matrix

These color matrices are for transforming between sensor values and XYZ values. When I first tried applying the matrix, image still looked strange and green until I got that these matrices are meant for converting from XYZ to sensor values. To get sensor->XYZ conversion, we need an inverted matrix!

Inverted matrix for conversion sensor values -> XYZ:
0.863 0.305 -0.033
0.257 0.793 -0.058
-0.16 -0.268 1.39

After applying this matrix, image still looks a bit strange, that's because we still need to apply XYZ -> sRGB transform to properly view it on computer or broadcast monitor.

Matrix for conversion from CIE-XYZ to Linear D65 sRGB:
3.24081 -1.53731 -0.498587
-0.969243 1.87597 0.0415551
0.0556384 -0.204007 1.05713

After these conversions we get the nice linear image in sRGB space that we can grade.

Sample images (with sRGB gamma applied for viewing purposes!)

These samples are images after conversion to sRGB and gamma transform. It is obvious that they have some differences compared to the way Resolve or Lightroom present the dng-s. Values seem to be more evenly distributed and image is somewhat flatter. There also is some green cast in first shots. This is because no G/M tint correction has been applied yet.

 
Cineform debayer algorithms

Following are samples from shot 1 with different debayer settings left-right, top-bottom:

Bilinear demosaic
Matrix 5x5 adaptive
CF Advanced smooth
CF Advanced detail 1
CF Advanced detail 2
CF Advanced detail 3

BMCC sample clips from Afterglow footage

Some sample footage from John Brawley's Afterglow footage have been released. They are DNG files containing camera raw data and are available for download through this post from John's blog HERE.

So, here is my take on grading the raw footage:
Graded in Lightroom because I wanted to try new Lightroom and see what I can do with dng files in there. Actually I'm not too sure I could make the same grade in Resolve because I'm not that used to Resolve yet.

Footage itself is great, some indoor shots, outdoor shot and low-light shots which show how much or little grain you get. Sensor white balance is I think 5600K and evening shots were very warm orangy due to that. I changed WB in Lightroom quite a bit, down to 3100-3200K. For shot 3, balcony, image looked too cool so I pushed WB up to 7200K. For first scenes I lowered the exposure and highlight values to get face back to normal exposure. It's rather amazing what these raw files are capable of! Very nice detail, dynamic range and little noise. Even the last two shots that were shot in low light situation and needed heavy WB correction stayed together very well and noise is not a big issue there.

Overall look - definitely impressive for 3000$ camera!

Something to listen

Since my mobile phone decided that some buttons are useless and should not work, I changed my service provider to get cheaper calls and free new phone. New phone will still be phone sized not a shiny bread board that talks but it has some new bells and whistles my previous phone didn't have. One of them is music player that, in addition to music, can also spend my time with some educational listening. So I decided to download...

The entire FXGuide audio archive

Every single piece of audio from FX Podcast, Red Centre and VFX Show are now downloaded totalling 10.4 GB worth of mp3 files. It should be about 360 hours of listening, more than enough for every bus, train or plane trip in foreseeable future.

When this phone decides to stop working, phones with nice screens and quality video players probably cost next to nothing. Then I will download all the FXGuide TV episodes!

Image debayering - what and how?

A recent thread about whether Blackmagic camera will have an antialiasing filter or not has gotten me thinking about the process of debayering/demosaicing in general and it's applications for different purposes. Currently topics discussed in that tread are related to whether or not BMCC will have an AA filter, what would be the consequences of having it or not and what is the purpose of such filter in general. I suggest reading that thread, lots of interesting stuff being written!

So, what is debayering / demosaicing?

Almost all still cameras and most video cameras these days have the so-called CMOS sensor. It means that light is captured on single sensor (as opposed to CCD systems which have separate sensor for each color) and to get color information out of it, a special filter, called Color Filter Array (CFA) is placed on the sensor. This filter has alternating pattern of red, green and blue sites in checkerboard (mostly) or some other pattern (hexagonal patterns are sometimes used). Because human eye is most sensitive to green light and thus green gives us lightness information, CFA has as much green sites as red and blue combined. In 2x2 checker, two diagonal patches would be green and other ones red and blue. Bayer pattern is essentially a hard-wired 4:2:2 subsampling schema that operates on RGB channels (in GRB order in this case).

CMOS sensor itself is basically black and white, it does not "see" color but only the intensity of light. So through filter we get intensities of green, red and blue light in a certain pattern. To get full RGB raster from this intensity map, debayering operation is performed. In it's simplest form, average values of adjascent sites can be calculated to make full image for each color. Simple averaging is easy but tends to give a very low quality image. For better results, "smarter" algorithms can be used that take into account value gradients, similarities and what not. A lot of these algorithms are patented and secret because among digital image makers, better image is worth a lot of money. Some information about different algorithms can be found on Wikipedia

What about BMCC

One topic that got me thinking about the difference between combating moire and aliasing on DSLRs and digital film cameras was the AA filter from company called Mosaic Engineering. They offer a filter that you can attach in front of Canon 7D or 5D  sensor and get rid of ugly color moire issues. In the thread mentioned before, some people argued that not having such a filter in front of the BMCC sensor would turn up similar ugly patterns as on most DSLRs. I can't say that I totally get the logic behind this. DSLRs generally skip lines to pull HD image from 5-6K sensor without downscaling that is some expencive processing. Skipping lines makes images vulnerable to moire because when pattern frequency and line skipping frequency clash, moire jumps up.

In the Pool Shark clip from John Brawley, chair with double fibre mesh backing made people jump because "BMCC has moire!" although this pattern is visible for anyone who sees such chair in person. Fibre mesh on the back of chair is somewhat similar to sensor that skips elements. Some spots are "blind", some "see". When this blind-see pattern sees another high-contrast pattern, these blind-see spots can either line up or not. When they don't some spots see another pattern, some dont and new pattern is formed that has maximums at lined up parts and minimums where patterns "cancel" each other out. Like two sine waves summed. This happens with sensors that have gaps between image elements, whatever the reason for this is (skipping, gaps due to sensor construction etc).

BMCC sensor does not (?) have gaps between sensor elements, at least I hope so. Gapless sensor catches all light that falls on it and thus should effectively prevent most moire patterns. It is natural antialiasing where for example a very thin diagonal line leaves mark on all pixels it crosses, although it doesn't cover any of them entirely. The intensity of pixel gives us impression that line crosses it but only barely and thus resulting image does not show aliasing. Aliased image would be produced if we tested for stick only in the dead centers of elements. Most pixels would show nothing and some would show everythin, thus a jagged aliased image.

What happens with this naturally antialiased bayer image during demosaicing is totally another story and this is where most of the ugly colored 1-2 pixel wide edges appear. They are the result of incorrect calculations in predicting what RGB values in this point should be. From totally black and white source image projected on bayer sensor, demosaicing can produce a rather rainbow like result where edges shimmer with purple, yellow, blue and every other color imaginable.

New understanding!

During writing the above text, it clicked that even with gapless sensor, patterns can introduce moire because for example on one element, 100% value is captured but on other two only 50% each because light patch falls on both of them. This leads to bright values on every third element and half-bright on every other two and so a new pattern that didn't exist in projected image appears as moire

This is where AA filter comes to play and cuts off higher frequencies that produce most of such moire.