I am working on android stack since cupcake release. My
experience with android is extensive and worked across multiple layers of
android stack right from application to application framework, to middleware to
HAL and Drivers. In this blog, I want to
share some details of android media framework and camera architecture.
- Android Libstagefreight media framework flow
This section gives the overview about the google libstagefright
multimedia framework overview and flow.
- Media server starts
the media services such as camera service, media player service, audio
flinger and audio policy service.
- Media player service
in turn registers with the Binder for Inter process/thread communication.
- Media player->media
player service -> instantiates the required framework
- StagefrightPlayer
instantiated from the media player service, creates the graph Awesome
player and registers listener callback
- Awesome player gets
the OMX Client , instantiates the OMX Master ,register OMX Vendor plugins
- Register the Sniffer
function for each file container and push it in the Queue. The
snifferfunction can read and give the basic information about the meta
information
- Register Event
callback for Video ,Audio status ,End of stream and buffer event.
- Reset the entire path
, change the flag state to idle.
- Start the timed event
queue in separate thread loop. It processes the event from the queue and call
the appropriate function to process the request.
- Set data source, set
Video surface values are stored in the Awesome player path
- During onprepare Async -> post the
event , come back to application
- On receiving prepare
event , event thread calls the appropriate function to process the request
- On the Prepare Async
function
-
complete the Data Source component creation
(http or rtsp or file format)
-
Incase of the http or rtsp component, it starts
the event loop , start connection the port
- Parse the basic atom
and find the mime type. Create media extractor for reading the data .
- gets the number of
track ,get meta data for each track
- Create media data
extractor for each track and set Audio ,Video Source for the media path
- Get the matching OMX
Component from the OMX repository
- Instantiate Codec if
software codec required
- Allocate the OMX
Component using Binder Client- Server interface
- Allocate Node instance ->OMX Component
-> init video Decoder
- Init video decoder
,allocates the buffer ,sends the command to OMX Component
- Data source allocates
the required buffer.
- Same is repeated (init
,start)for audio path also
- Once init is done,
awesome player posts the buffering event for processing the buffer
requests.
- OnBufferEvent captures
the event , process accordingly .If graph is in prepare state changed to
prepared state and calls the PrepareAsync to complete
- When the graph gets
play command from the application ,Awesome player creates the Audio player
- Camera Architecture
This section captures the
overall flow of the Android Camera in the user space and in the kernel space .
It describes about the HAL implementation of the Camera and user space to
kernel space communication. It also describes the call flow involved for
various use cases like Camera init , Preview and Capturing.
Camera Application: This is the Android Camera Default
application and can be replaced in future with other customized application. Other
applications (Video Record, Image Capturer etc..) can use the Camera
Application API’s.
Camera JNI: This is the JNI interface for the Camera Application.
It converts Java function Calls into native calls.
Camera Interface: The camera interface
(camera.cpp) is proxy for remote object in camera service layer. It is the
middle level interface between JNI and Camera Binder.
Camera Binder Interface: ICamera is the Binder interface for IPC call and exposes the API’s for
Camera Interface. It binds with the Camera Service Layer for the Communication.
Camera service: As part of system boot the
camera service is started by the media server. It act as the mediator between
Application and Hardware Layer to change the context. Application gets the
Camera service handle and starts communicating. The camera service transforms
the Application commands to the Hardware and sends the reply back. It interacts with Surface Flinger for
delivering preview frames on display.
Android
Camera HAL Interface: HAL for Android system to interact
with the Hardware Layer. It is required because not all the hardware Driver
exposes the standard API’s for interaction. This makes easy porting with
different Hardware This is the first level HAL implementation as part of opensource.
Vendor Camera HAL Interface: HAL for actual camera
hardware Vendors’s HAL implements Google HAL APIs to hook up Vendor native
camera driver in kernel.
Camera Driver: Camera HAL invoke Native
Camera driver in Kernel space. User Space threads are managed by Vendor HAL and
Native Camera driver . Camera Driver interacts with User space HAL Layer and
Kernel Driver. It implements the Device Node for Control, Config and Frame.
1. Battery Service Start up and Event Registration.
2. Native Battery Change
Event Notification:
3. Receiving Battery
information
4. Broadcast to the Application:
The system
server loads the system library (android_servers) and starts all the Android
services such as telephony manager, battery service, light service and
Bluetooth. Start up of each service is currently written directly into the run
method of the system server.
As mention in the figure ,the battery Service follows the
below steps for registration
1.
It opens the Power Supply directory path
(/sys/class/power_supply)
2.
Reads the information from the file in the path
and update the path information as below
"/sys/class/power_supply/ac/online"
"/sys/class/power_supply/usb/online"
"/sys/class/power_supply/battery/status"
"/sys/class/power_supply/battery/health"
"/sys/class/power_supply/battery/present"
"/sys/class/power_supply/battery/capacity"
"/sys/class/power_supply/battery/batt_vol"
"/sys/class/power_supply/battery/batt_temp"
"/sys/class/power_supply/battery/technology"
Whenever the application requests for the Battery
information, the corresponding JNI Layer
(com_android_server_BatteryService.cpp) reads the information from the path
specified and returns to the application.
The system server starts the battery service and adds into
service manager list. Battery Service Register the UEVENT with power_supply for
Event notification. It calls the JNI to update the initial Battery
information.
2.Native Battery Change Event Notification
The battery service receives the battery change
event notification via UEvent.
1. The Battery Service is registered as a uevent Observer
(mUEventObserver.startObserving ("SUBSYSTEM=power_supply");)
2. When the battery status changed and uevent will
be generated from driver to the user level
3. When the power_supply related uevent happens,
the onUevent method of Observer will be called.
4.The update function in battery service is
responsible to read the info and broadcast to the Application
3. Receiving Battery information
Application receives the Battery
update information through Uevent Observer. It calls JNI to get the latest battery
information
1. Power supply core calls user space event observer to
notify the battery change event.
2. The registered UEvent observer informs to the battery
service.
3. Battery service reads
all the latest battery information from Sys file via JNI and broadcast to the
application.
A.
Charging
B.
Discharging
C.
Not Charging
D.
Battery Full
E.
Unknown
There are five different battery health categories
- good
- Dead
- Overheat
- Over
Voltage
- unknown
4. Broadcast to the
Application
Once the Battery Service receives the battery information it
broadcast the status to the Application.
1. It broadcast the intent ACTION_REQUEST_SHUTDOWN if there
is no power in the battery.
2. It broadcast the intent ACTION_REQUEST_SHUTDOWN if the
temperature is too high.
3. It broadcast varies intent such BATTERY_LOW, BATTERY_OK,
BATTERY_CHANGED to the application
4. Finally it packs all the Battery
information and broadcast to everyone in the application space.
