Camera2 Flash: The Complete Guide (With Working Sample App)
The Camera2 API is notoriously hard to use, not very well documented, and official sample code provided by Google is more than 7 years old. Implementing a functional flash procedure is what I have spent the past few weeks on, and the journey has been difficult. This post tries to demystify the usage of Camera2, particularly in the context of modern Kotlin where not as many examples exist. The example app shows a basic implementation of the Camera2 API focusing on taking pictures with flash enabled.
Resources
The code is available here
The premise
Anyone who has spent some time using the Camera2 API already knows that it is not well documented, confusing, and full of potential edge cases and race conditions. This is confirmed by looking at example projects on the official Camera2 site, where most of them are older than 5 years. Managing all resources and timing everything correctly is hard. As an example, consider taking pictures with a flash. The whole procedure needs to be managed manually:
Start the precapture sequence that turns on the flash and sets exposure, white balance and focus
Once the procedures have converged, we are ready to take a picture, the flash fires once more and the picture is taken during that moment
As you can imagine, correctly timing this procedure can be challenging and error-prone. This is further complicated by different HAL implementations on the many Android devices that exist out there. There are some implementations that utilize the flash, but trying to follow them is difficult and the code is outdated and potentially not free of race conditions. This blog post explains how we can successfully implement a working flash procedure using a more modern approach with CompletableDeferred from Kotlin coroutines.
Camera2 basics
Before talking about the flash procedure, I will briefly cover the basic idea of how Camera2 works. I will not go in-depth on this, but the example app documents the whole setup procedure in CameraFragment.kt. I tried keeping the setup as minimal as possible, so it's only displaying a preview and allows the user to take a picture. You can reference the code for a basic setup, which should be expandable if you need any custom features.
At the core of the Camera2 API is the CaptureRequest. According to the docs, a CaptureRequest is «An immutable package of settings and outputs needed to capture a single image from the camera device». A CaptureRequest is created using a builder and dispatched using either capture() or setRepeatingRequest(). The two main components of the request are its target, which specifies what output surfaces this relates to, and its configuration, which specifies what settings the camera should use for this request. For example, to capture a still image with flash, we use an ImageReader as the target surface and enable flash using the CONTROL_AE_MODE_ON_ALWAYS_FLASH setting. The request would then be dispatched using capture(). If we wanted to show the user a preview of what the camera is seeing and enable autofocus, we would use a TextureView as the target and use CONTROL_AE_MODE_ON to enable autofocusing. Instead of using capture() to dispatch this request, which would only dispatch it once, we use setRepeatingRequest(), which tells the camera to continuously capture images using the settings provided.
CaptureRequests can be interleaved. This means that we can show a preview to the user using setRepeatingRequest() and then when the user presses a button capture an image by dispatching a capture() request.
Implementing a robust flash procedure
Now that we have seen the basic workings of the Camera2 API, let's talk about implementing a robust flash procedure. I have spent many hours trying to get this to work and finally managed by using the following guide and adapting it to a normal flash procedure instead of screen flashing. Our implementation makes use of two main components: CompletableDeferred and CameraCaptureSession.CaptureCallback.
Completable Deferred
Completable Deferred is a neat feature from Kotlin's Coroutines. It is a future which can be completed using functions complete() or cancel(). Take a look at the example code below:
The output of the code above would be:
The main advantage here is that we are able to manually control when the task should be completed. This is very useful when we want to wait for the camera to reach a certain state.
CameraCaptureSession.CaptureCallback
When setting up a setRepeatingRequest() we can add different types of callbacks. The one that will be useful for us is onCaptureCompleted(), which is called every time a capture has been performed. Since we are using repeating requests, this essentially happens each frame. We can leverage this callback to repeatedly check which state our camera is in. This allows us to check for convergence of auto exposure and white balance before capturing with flash.
The flash procedure
Now let's get to the fun part. At the heart of the flash procedure is the repeatingCaptureCallback:
This callback allows us to do two things: Using awaitAeModeUpdate(targetAeMode: Int) we can specify a specific auto exposure mode we want to wait for. Once it has been reached, the deferred is completed so we can listen to it and wait for the correct state. This can be expanded to not only listen for a specific AE mode — in theory you could wait for any specific condition on the camera. Using awaitAeAwbConvergence() we can wait for auto exposure and white balance to converge. Once they are in a converged state, the deferred is completed and we can continue execution. Now taking a picture is just a matter of orchestrating everything together nicely:
The first thing to note here is that we are reusing our capture request builder to turn on flash mode. Secondly, it's important to start the whole procedure in a coroutine to avoid blocking the main thread. If we were to launch this on the main thread, the preview would freeze until the picture has been taken. As you can see, we are also utilizing awaitAeModeUpdate() to wait until the flash mode has been enabled. Once that is the case, we run the precapture sequence (below), after which we perform a simple capture() request to take our image. Finally, we reset the auto exposure mode back to default to prevent the flash from being active all the time.
The precapture sequence is what triggers our metering, waits until convergence, and then continues.
That's it. With this you should be able to perform a flash sequence and avoid all the headache of correctly timing all the individual steps.
Conclusions
While the Camera2 API is notoriously difficult to work with, once you get the hang of it the components start to click. Using the modern approach with completable deferreds additionally simplifies synchronization of different tasks and could be applied to many other use cases.
