Android Stack


I am working on android stack since cupcake release. My experience with android is extensive and worked across multiple layers of android stack right from application to application framework, to middleware to HAL and Drivers.  In this blog, I want to share some details of android media framework and camera architecture.

  • Android Libstagefreight media framework flow

This section gives the overview about the google libstagefright multimedia framework  overview and flow.
  1. Media server starts the media services such as camera service, media player service, audio flinger and audio policy service.
  2. Media player service in turn registers with the Binder for Inter process/thread communication.
  3. Media player->media player service -> instantiates the required framework
  4. StagefrightPlayer instantiated from the media player service, creates the graph Awesome player and registers listener callback
  5. Awesome player gets the OMX Client , instantiates the OMX Master ,register OMX Vendor plugins
  6. Register the Sniffer function for each file container and push it in the Queue. The snifferfunction can read and give the basic information about the meta information
  7. Register Event callback for Video ,Audio status ,End of stream and buffer event.
  8. Reset the entire path , change the flag state to idle.
  9. Start the timed event queue in separate thread loop. It processes the event from the queue and call the appropriate function to process the request.
  10. Set data source, set Video surface values are stored in the Awesome player path
  11. During onprepare Async -> post the event , come back to application
  12. On receiving prepare event , event thread calls the appropriate function to process the request
  13. On the Prepare Async function
-        complete the Data Source component creation (http or rtsp or file format)
-        Incase of the http or rtsp component, it starts the event loop , start connection the port
  1. Parse the basic atom and find the mime type. Create media extractor for reading the data .
  2. gets the number of track ,get meta data for each track
  3. Create media data extractor for each track and set Audio ,Video Source for the media path
  4. Get the matching OMX Component from the OMX repository
  5. Instantiate Codec if software codec required
  6. Allocate the OMX Component using Binder Client- Server interface
  7. Allocate Node instance ->OMX Component -> init video Decoder
  8. Init video decoder ,allocates the buffer ,sends the command to OMX Component
  9. Data source allocates the required buffer.
  10. Same is repeated (init ,start)for audio path also
  11. Once init is done, awesome player posts the buffering event for processing the buffer requests.
  12. OnBufferEvent captures the event , process accordingly .If graph is in prepare state changed to prepared state and calls the PrepareAsync to complete
  13. When the graph gets play command from the application ,Awesome player creates the Audio player      

  •  Camera Architecture

This section captures the overall flow of the Android Camera in the user space and in the kernel space . It describes about the HAL implementation of the Camera and user space to kernel space communication. It also describes the call flow involved for various use cases like Camera init , Preview and Capturing.




Camera Application: This is the Android Camera Default application and can be replaced in future with other customized application. Other applications (Video Record, Image Capturer etc..) can use the Camera Application  API’s.

Camera JNI: This is the JNI interface for the Camera Application. It converts Java function Calls into native calls.


Camera Interface: The camera interface (camera.cpp) is proxy for remote object in camera service layer. It is the middle level interface between JNI and Camera Binder.


Camera Binder Interface: ICamera is the Binder interface for IPC call and exposes the API’s for Camera Interface. It binds with the Camera Service Layer for the Communication.


Camera service: As part of system boot the camera service is started by the media server. It act as the mediator between Application and Hardware Layer to change the context. Application gets the Camera service handle and starts communicating. The camera service transforms the Application commands to the Hardware and sends the reply back. It interacts with Surface Flinger for delivering preview frames on display.


Android Camera HAL Interface: HAL for Android system to interact with the Hardware Layer. It is required because not all the hardware Driver exposes the standard API’s for interaction. This makes easy porting with different Hardware This is the first level HAL implementation as part of opensource.
Vendor Camera HAL Interface: HAL for actual camera hardware Vendors’s HAL implements Google HAL APIs to hook up Vendor native camera driver in kernel. 

Camera Driver: Camera HAL invoke Native Camera driver in Kernel space. User Space threads are managed by Vendor HAL and Native Camera driver . Camera Driver interacts with User space HAL Layer and Kernel Driver. It implements the Device Node for Control, Config and Frame. 
  • Battery Service 
Battery service provides information such as battery health, capacity, status, voltage and temperature. The following sequence of operations are executed for the Battery Information check and update

1. Battery Service Start up and Event Registration.
2. Native Battery Change Event Notification:
3. Receiving Battery information
4. Broadcast to the Application:

1. Battery Service Start up and Event Registration

       The Zygote is launched by the init process and will basically just start executing and initialize the Dalvik VM. It also starts the system server. It is the first java component to run in the system.
      The system server loads the system library (android_servers) and starts all the Android services such as telephony manager, battery service, light service and Bluetooth. Start up of each service is currently written directly into the run method of the system server.
       DVM calls JNI load while loading the system library to start the basic native services such as battery service, vibrator Service, alarm service and sensor Services.

As mention in the figure ,the battery Service follows the below steps  for registration
1.     It opens the Power Supply directory path (/sys/class/power_supply)
2.     Reads the information from the file in the path and update the path information as below
      "/sys/class/power_supply/ac/online"
      "/sys/class/power_supply/usb/online"
       "/sys/class/power_supply/battery/status"
       "/sys/class/power_supply/battery/health"
       "/sys/class/power_supply/battery/present"
       "/sys/class/power_supply/battery/capacity"
                           "/sys/class/power_supply/battery/batt_vol"
                           "/sys/class/power_supply/battery/batt_temp"
 "/sys/class/power_supply/battery/technology"
      
Whenever the application requests for the Battery information, the corresponding JNI Layer (com_android_server_BatteryService.cpp) reads the information from the path specified and returns to the application.

The system server starts the battery service and adds into service manager list. Battery Service Register the UEVENT with power_supply for Event notification. It calls the JNI to update the initial Battery information. 




2.Native Battery Change Event Notification

The battery service receives the battery change event notification via UEvent.

1. The Battery Service is registered as a uevent Observer (mUEventObserver.startObserving ("SUBSYSTEM=power_supply");)
2. When the battery status changed and uevent will be generated from driver to the user level
3. When the power_supply related uevent happens, the onUevent method of Observer will be called.
4.The update function in battery service is responsible to read the info and broadcast to the Application



3. Receiving Battery information

Application receives the Battery update information through Uevent Observer. It calls JNI to get the latest battery information

1. Power supply core calls user space event observer to notify the battery change event.
2. The registered UEvent observer informs to the battery service.
3. Battery service reads all the latest battery information from Sys file via JNI and broadcast to the application.

Battery status are classified as
A.    Charging
B.    Discharging
C.    Not Charging
D.    Battery Full
E.    Unknown

There are five different battery health categories

  1. good
  2. Dead
  3. Overheat
  4. Over Voltage
  5. unknown



4. Broadcast to the Application

Once the Battery Service receives the battery information it broadcast the status to the Application.
1. It broadcast the intent ACTION_REQUEST_SHUTDOWN if there is no power in the battery.
2. It broadcast the intent ACTION_REQUEST_SHUTDOWN if the temperature is too high.
3. It broadcast varies intent such BATTERY_LOW, BATTERY_OK, BATTERY_CHANGED to the application
4. Finally it packs all the Battery information and broadcast to everyone in the application space.




at
Labels:

2 comments:

  1. Ganesh21 May 2018 at 01:02

    Very informative data. The author seem to have vast experience in the domain. Keep sharing the good knowledge!

    ReplyDelete
  2. Unknown21 May 2018 at 02:03

    Excellent Info.Please share us more.

    ReplyDelete

Subscribe to: Post Comments (Atom)

IoT Device Communication protocols It is essential to have secure, optimized data delivery between IoT Devices, Gateway and cloud edge...

  • C and C++ Static and Dynamic Code Analysis- Code Quality improvement
    Writing good quality code is the goal of all the software developers. However, writing very good quality code following complete rules at...
  • (no title)
    IoT Device Communication protocols It is essential to have secure, optimized data delivery between IoT Devices, Gateway and cloud edge...
  • Android Stack
    I am working on android stack since cupcake release. My experience with android is extensive and worked across multiple layers of andro...