Sep 6, 2014

What I've learned about MMAL and picamera: Raspberry Pi drive recorder with GPS logger

To interact with Raspberry PI Camera Board from Python, I chose a library called picamera. It provides us interface to control Raspberry Pi Camera without using raspistill, raspivid or raspiyuv utilities. Actually it provides even more features such as taking images and recording video at the same time, modifying options while recording, etc. I think this library is reliable because it is well maintained by its owner, waveform80, and is used in adafruit's sample project. While reading its code, I found it was very important to understand the concepts and basics of Multi-Media Abstraction Layer (MMAL), so I began reading MMAL's specification. I am going to introduce what I have learned about MMAL and how picamera works with it.

MMAL

First of all, Raspberry Pi has BCM2835 that has VideoCore IV inside. As I already mentioned, MMAL stands for Multi-Media Abstraction Layer, and it provides lower level interface to multi-media components running on this VideoCore thing. Actually MMAL runs over OpenMAX so there is another layer below MMAL, but I decided to ignore it. As long as MMAL works fine as an abstraction layer and fulfil my requirement, I do not have to be aware of anything behind it.

Here is how MMAL works.
  1. Client creates MMAL component via API.
  2. When creation is done a context is returned to component.
  3. This context provides at least one input port and one output port. This number and format may vary depending on what this component represents. The format of input port must be set when component is created; The format of output port may be decided later and can be modified by client.
  4. Client and component send buffer headers to each other through input/output port to exchange data. Component can also exchange buffer headers with other connected component.
  5. There are two kinds of buffer headers:
    • One contains meta data and pointer to actual data.
      Payload, the actual data to be exchanged, is not stored in buffer header so the memory can be allocated outside of MMAL.
    • The other one contains event data.
  6. These buffer headers are pooled and each of them gets recycled when it is no longer referenced.

How picamera works with MMAL

The good thing about MMAL is that MMAL components can be connected to each other so they can exchange buffer headers. This well designed library, picamera, creates and uses some different kinds of components effectively to work with Raspberry Pi Camera Board. My favorite thing is that it creates a splitter component that receives video output from camera component and splits this input to 4 output ports. This way we can capture image via video port while video recording is in progress.

Components created on initialization

When picamera is initialized, it creates components below:

Camera component

This is the most important one. Everything starts from here. This component provides 3 output ports:
  • preview
  • video
  • still
The other components I am going to introduce receive data directly or indirectly from these output ports.

Preview component

Basically we have two components here.
One of these components' input port is connected to camera component's preview output. On initialization and when actual preview is not needed, null-sink component is connected. It is necessary because if we do not connect any component to the preview output provided by camera component, the camera doesn't measure exposure and captured images gradually fade to black. When start_preview() is called, it creates preview renderer component and replaces null-sink component with this preview component that actually shows preview.

Splitter component

Its input port is connected to the camera component's video output. It has 4 output ports and each output port works as a copy of camera component's video output. This way, we can record data from one of its output ports and capture image via another output port at the same time. Yes, capturing image via video port is faster and you might want to use it (camera.capture('foo.rgb', use_video_port=True)).

Other components

Some other kinds of components including encoder and renderer are created when necessary. Thanks to waveform80, A new version of picamera, ver. 1.8, was released today and it provides interface to create and manage MMAL renderer components. This must be really handy when you want to add overlay to your preview. I, for one, wanted to have this feature so badly that I asked @wafeform80 about release schedule, last night.
He was nice enough to respond to me with a sense of humor and he really released it!
I am going to rewrite the live preview part and try its new features over this weekend.
EDIT: Follow-up article is now available.

Posted by at
Labels: ,