Tuesday, August 28, 2012

BMCC RAW to Cineform RAW

EDIT: useful links
http://lclevy.free.fr/cr2/
http://www.guillermoluijk.com/tutorial/dcraw/index_en.htm

One of the hot topics in BMCuser forum is the Cineform Conversion and particularly the raw dng to Cineform raw possibility. As Cineform can convert dpx files to cineform raw files (with dpx2cf command line utility), this could be one way to deal with large file sizes that working with raw files tends to bring upon.

NOTE: Older Cineform versions don't seem to support dng conversion, you need release from at least april/may of 2012! Otherwise you get the "DPX file not found" error.

DNG sequence to Cineform RAW file

Converting dng sequence to cineform is pretty easy. Use dpx2cf tool with, for example, following command:

dpx2cf.exe Frame*.dng shot1_cf.avi -q5

This reads all dng files from shown source (* marks frame numbers) and converts it into cineform file. Additional settings like frame rate etc can be set, see settings by simply running the program in command prompt.

EDIT: there was a typo in command example, there should not have been a " mark after shot1_cf.avi.

Cineform RAW looks green?

After conversion it is immediately apparent that something is wrong. Image looks very green and strange. This is because Cinefrom converter does not apply any correction on the image and we see pure raw (debayered ofcourse) image. As sensors are usually more sensitive to green light, image looks green.

Raw processing software (like Lightroom or Camera Raw) applies camera calibration correction before showing the image to user and so we don't really know what the sensor actually sees. Because dng and other raw formats contain these profiles in their metadata, raw conversion software usually knows how to perform this correction. Cineform converter on the other hand does not use this info, puts raw sensor values straight into the file and also shows us these values without correction.

Reading DNG metadata

To obtain metadata from dng file, we need some software that can extract that info from the file header. For example a handy set of tools called EXIFUtils. With this tool we can extract all the interesting metadata that BMCC dng files have to offer.

For example:

Camera
  image-type                    : Main
  main-width                    : 2432
  main-len                      : 1366
  main-bits-sam                 : 12
  main-comp                     : None
  photo-int                     : Color Filter Array (CFA)
  strip-off                     : 9856
  orient                        : Upper Left
  sample-pix                    : 1
  row-strip                     : 1366
  strip-cnt                     : 4983168
  planar-conf                   : Chunky
  cfa-rpt-pat-dim               : 2 2
  cfa-rpt-pattern               : 01000201
  tag-9216                      : 00000001
  dng-version                   : 01020000
  dng-model                     : Blackmagic Cinema Camera
  dng-lin-table                 : 54 55 55 56 57 58 59 59 60 61 62 63 63 64 65
                                  66 66 67 68 69 70 70 71 72 73 74 74 75 76 77
                                  78 78 79 80 81 81 82 83 84 85 85 86 87 88 89
                                  89 90 91 92 93 93 94 95 96 96 97 98 99 100
                                  100 101 102 103 104 104 105 106 107 107 108
                                  109 110 111 111 112 113 114 115 115 116 117
                                  118 119 119 120 121 122 122 123 124 125 126
                                  126 127 128 129 130 130 131 132 133 134 134
                                  135 136 137 137 138 139 140 141 141 142 143
                                  144 145 145 146 147 148 149 149 150 151 152
                                  152 153 154 155 156 156 157 158 159 160 160
                                  161 162 163 164 164 165 166 167 167 168 169
                                  170 171 171 172 173 174 175 175 176 177 178
                                  179 179 180 181 182 182 183 184 185 186 186
                                  187 188 189 190 190 191 192 193 194 194 195
                                  196 197 197 198 199 200 201 201 202 203 204
                                  205 205 206 207 208 209 209 210 211 212 212
                                  213 214 215 216 216 217 218 219 220 220 221
                                  222 223 223 224 225 226 227 227 228 229 230
                                  231 231 232 233 234 235 235 236 237 238 238
                                  239 240 241 242 242 243 244 245 246 246 247
                                  248 249 250 250 251 252 253 253 254 255 256
                                  257 257 258 259 260 261 261 262 263 264 265
                                  265 266 267 268 268 269 270 271 272 272 273
                                  274 275 276 276 277 278 279 280 280 281 282
                                  283 283 284 285 286 287 287 288 289 290 291
                                  291 292 293 294 295 295 296 297 298 298 299
                                  300 301 302 302 303 304 305 306 306 307 308
                                  309 310 310 311 312 313 313 314 315 316 317
                                  317 318 319 320 321 321 322 323 324 325 325
                                  326 327 328 328 329 330 331 332 332 333 334
                                  335 336 336 337 338 339 339 340 341 342 343
                                  343 344 345 346 347 347 348 349 350 351 351
                                  352 353 354 354 355 356 357 358 358 359 360
                                  361 362 362 363 364 365 366 366 367 368 369
                                  369 370 371 372 373 373 374 375 376 377 377
                                  378 379 380 381 381 382 383 384 384 385 386
                                  387 388 388 389 390 391 392 392 393 394 395
                                  396 396 397 398 399 399 400 401 402 403 403
                                  404 405 406 407 407 408 409 410 411 411 412
                                  413 414 414 415 416 417 418 418 419 420 421
                                  422 422 423 424 425 426 426 427 428 429 429
                                  430 431 432 433 433 434 435 436 437 437 438
                                  439 440 441 441 442 443 444 444 445 446 447
                                  448 448 449 450 451 452 452 453 454 455 455
                                  456 457 458 459 4
  dng-bl-lvl-dim                : 1 1
  dng-bl-level                  : 256
  dng-wh-level                  : 60074
  dng-def-crop-origin           : 16 8
  dng-def-crop-size             : 2400 1350
  dng-color-matrix-1            : 1.31 -0.50 0.01 -0.42 1.44 0.05 0.07 0.22 0.
                                  73
  dng-color-matrix-2            : 1.01 -0.27 -0.08 -0.49 1.34 0.11 -0.06 0.33
                                  0.54
  dng-camera-calib-1            : 1.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 1.00
  dng-camera-calib-2            : 1.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 1.00
  dng-as-shot-neutral           : 0.63 1.00 0.79
  dng-base-exp                  : 2.40
  dng-calib-illum-1             : 17
  dng-calib-illum-2             : 21
  tag-c763                      : 0859011400000000
  tag-c764                      : 24.00

We get a lot of info about file size, black levels, crops etc. And some color matrices.

DNG color matrix

These color matrices are for transforming between sensor values and XYZ values. When I first tried applying the matrix, image still looked strange and green until I got that these matrices are meant for converting from XYZ to sensor values. To get sensor->XYZ conversion, we need an inverted matrix!

Inverted matrix for conversion sensor values -> XYZ:
0.863 0.305 -0.033
0.257 0.793 -0.058
-0.16 -0.268 1.39

After applying this matrix, image still looks a bit strange, that's because we still need to apply XYZ -> sRGB transform to properly view it on computer or broadcast monitor.

Matrix for conversion from CIE-XYZ to Linear D65 sRGB:
3.24081 -1.53731 -0.498587
-0.969243 1.87597 0.0415551
0.0556384 -0.204007 1.05713

After these conversions we get the nice linear image in sRGB space that we can grade.

Sample images (with sRGB gamma applied for viewing purposes!)

These samples are images after conversion to sRGB and gamma transform. It is obvious that they have some differences compared to the way Resolve or Lightroom present the dng-s. Values seem to be more evenly distributed and image is somewhat flatter. There also is some green cast in first shots. This is because no G/M tint correction has been applied yet.





























 
Cineform debayer algorithms

Following are samples from shot 1 with different debayer settings left-right, top-bottom:

Bilinear demosaic
Matrix 5x5 adaptive
CF Advanced smooth
CF Advanced detail 1
CF Advanced detail 2
CF Advanced detail 3

1 comment:

  1. Awesome! Let us know over at bmcuser when you figure out a one step solution for converting with these settings.

    One thing though. They seem to have a gamma curve applied.. a flatter image would be better.

    ReplyDelete