SPRO2004 - Advanced DataWrangling

DATAWRANGLING

Data Wrangling is the process and management of moving data from a media card to a designated folder structure on some form of permanent storage device for logistical and backup purposes.

This process creates a 'Virtual' media card that provides the same functionality and benefits as a media card (SD, SxS) when working in a digitial non-linear editing system.

Purposes for DataWrangling

  1. Move media files in a secure manner.

  2. Keep data safe and backed up.

  3. Enable multiple media import sessions.

  4. Simplify viewing dailies and making paper edits.

  5. Facilitate a faster and more structured editing process.

You should have already created a /wiki/spaces/CKB/pages/2583792 (This Root folder that will hold ALL the Raw Media (video and audio) for your production).

Now create a new folder for every card you backup/datawrangle:

yymmdd_UniqueIdentifer_CardNo.: These folders organise the Raw Media according to shoot date. It also adds the name of your production.

CamA: Holds the media from Camera A (if you are shooting multiple cameras)

  • RawA001: Holds the Raw Media (Video) fro Camera A (if shooting multiple cameras). The numbering system is sequential ACROSS shoot days - in this way the digits indicate the recording order of the media cards. The next folder will be called RawA002, even if it is from the following day.

Sound: Holds the Synch Audio cards recorded in a Dual System process.

  • SND001: Holds the Raw Media (Audio) from the /wiki/spaces/CKB/pages/2583926.The numbering system is sequential ACROSS shoot days - in this way the digits indicate the recording order of the media cards. The next folder will be called SND002, even if it is from the following day.

PLEASE READ ME - LOG Notes.txt: A text file with information on the shoot, such as Shoot Location, Camera Type for Camera A/B, Media Card Type, Sound Recorder Model, any issues/problems/incidents. In other words, any information that will be useful for the post-production team.

Modified using source material from: http://nofilmschool.com/2011/09/data-wrangling/ 

Media Organsiation

REMEMBER: Good media organisation is key to a successful post-production process.Organise media chronologically and ALWAYS keep a comprehensive Shot List:

  • One folder for every shoot day (yymmdd_Unique_Identifier)

  • Subfolders for each Camera (CamA, CamB) and/or for each media card (RawA001, RawA002 or Raw001, Raw002 etc.)

For a more comprehensive Data Wrangling Guide see here

Create Folder Structure

Locate the MEDIA folder you created in a previous step.

Inside this folder create a new folder called 'yymmdd_UniqueIdentifier_ShootDay'

Six-digit date (shootday) followed by a unique name and identifier that makes sense to you (e.g. 150402_Rambo_Day01).

This is the folder for the media from your first shoot day.

If your shoot stretches over multiple days you will make several folders, one for each shoot day, each with a unique number.

Inside the 'yymmdd_Unique_Identifier' folder create a folder called 'Raw001'. This is the folder for your first media dump.

If you have a second media card from the same shoot day, make a folder called 'Raw002'

If you have a third media card from the same shoot day, make a folder called 'Raw003'

When you make Raw folders on consecutive shoot days, number the Raw folders sequentially from the first one you created.

OROROROR

Once your folder structure has been completed it is time to copy your media from the card.

Locate the media card on your desktop, and double click it to open it up.

Inside you will see one or more folders/file items depending on what kind of camera card you are looking at.


In this example, an SD card from a /wiki/spaces/CKB/pages/2583699 will only have one file item called 'PRIVATE'.


If you have shot on S x S cards using the /wiki/spaces/CKB/pages/2583999 you want to copy the entire BPAV folder.


If you have shot on the /wiki/spaces/CKB/pages/2583531 you want to copy all the media files from the SSD Drive.


If you have shot with the /wiki/spaces/CKB/pages/2583951 you want to copy the DCIM and MISC folders.

If you have shot on the GoPro Camera you want to copy the DCIM and MISC folders.

Select and grab every folder/file item on the card (Cmd + a), copy (Cmd + c) and paste (Cmd + v) to the 'Raw001' folder you created earlier.

If you have a second media card from the same shoot day put this card in the 'Raw002' folder.

Follow on consecutively with additional media cards.


Do this to every card throughout your shoot period, ensuring that each card gets its own sequentially numbered 'Raw' folder.

For a more comprehensive overview of the Data Wrangling Procedure see here

DON'T FORGET TO MANAGE YOUR MEDIA