Camera2 Flash: The Complete Guide (With Working Sample App)
The Camera2 API is notoriously hard to use, not very well documented, and official sample code provided by Google is more than 7 years old. Implementing a functional flash procedure is what I have spent the past few weeks on, and the journey has been difficult. This post tries to demystify the usage of Camera2, particularly in the context of modern Kotlin where not as many examples exist. The example app shows a basic implementation of the Camera2 API focusing on taking pictures with flash enabled.
Resources
The code is available here
The premise
Anyone who has spent some time using the Camera2 API already knows that it is not well documented, confusing, and full of potential edge cases and race conditions. This is confirmed by looking at example projects on the official Camera2 site, where most of them are older than 5 years. Managing all resources and timing everything correctly is hard. As an example, consider taking pictures with a flash. The whole procedure needs to be managed manually:
Start the precapture sequence that turns on the flash and sets exposure, white balance and focus
Once the procedures have converged, we are ready to take a picture, the flash fires once more and the picture is taken during that moment
As you can imagine, correctly timing this procedure can be challenging and error-prone. This is further complicated by different HAL implementations on the many Android devices that exist out there. There are some implementations that utilize the flash, but trying to follow them is difficult and the code is outdated and potentially not free of race conditions. This blog post explains how we can successfully implement a working flash procedure using a more modern approach with CompletableDeferred from Kotlin coroutines.
Camera2 basics
Before talking about the flash procedure, I will briefly cover the basic idea of how Camera2 works. I will not go in-depth on this, but the example app documents the whole setup procedure in CameraFragment.kt. I tried keeping the setup as minimal as possible, so it's only displaying a preview and allows the user to take a picture. You can reference the code for a basic setup, which should be expandable if you need any custom features.
At the core of the Camera2 API is the CaptureRequest. According to the docs, a CaptureRequest is «An immutable package of settings and outputs needed to capture a single image from the camera device». A CaptureRequest is created using a builder and dispatched using either capture() or setRepeatingRequest(). The two main components of the request are its target, which specifies what output surfaces this relates to, and its configuration, which specifies what settings the camera should use for this request. For example, to capture a still image with flash, we use an ImageReader as the target surface and enable flash using the CONTROL_AE_MODE_ON_ALWAYS_FLASH setting. The request would then be dispatched using capture(). If we wanted to show the user a preview of what the camera is seeing and enable autofocus, we would use a TextureView as the target and use CONTROL_AE_MODE_ON to enable autofocusing. Instead of using capture() to dispatch this request, which would only dispatch it once, we use setRepeatingRequest(), which tells the camera to continuously capture images using the settings provided.
CaptureRequests can be interleaved. This means that we can show a preview to the user using setRepeatingRequest() and then when the user presses a button capture an image by dispatching a capture() request.
Implementing a robust flash procedure
Now that we have seen the basic workings of the Camera2 API, let's talk about implementing a robust flash procedure. I have spent many hours trying to get this to work and finally managed by using the following guide and adapting it to a normal flash procedure instead of screen flashing. Our implementation makes use of two main components: CompletableDeferred and CameraCaptureSession.CaptureCallback.
Completable Deferred
Completable Deferred is a neat feature from Kotlin's Coroutines. It is a future which can be completed using functions complete() or cancel(). Take a look at the example code below:
fun main() = runBlocking {
// Create a CompletableDeferred
val deferred = CompletableDeferred<String>()
// Launch a coroutine that waits for the result
launch {
println("Waiting for result...")
val result = deferred.await()
println("Got result: $result")
}
// Simulate some work
delay(2000)
// Complete the deferred with a value
println("Completing the deferred...")
deferred.complete("Task finished!")
}
The output of the code above would be:
Waiting for result...
Completing the deferred...
Got result: Task finished!
The main advantage here is that we are able to manually control when the task should be completed. This is very useful when we want to wait for the camera to reach a certain state.
CameraCaptureSession.CaptureCallback
When setting up a setRepeatingRequest() we can add different types of callbacks. The one that will be useful for us is onCaptureCompleted(), which is called every time a capture has been performed. Since we are using repeating requests, this essentially happens each frame. We can leverage this callback to repeatedly check which state our camera is in. This allows us to check for convergence of auto exposure and white balance before capturing with flash.
The flash procedure
Now let's get to the fun part. At the heart of the flash procedure is the repeatingCaptureCallback:
private val repeatingCaptureCallback = object : CameraCaptureSession.CaptureCallback() {
private var targetAeMode: Int? = null
private var aeModeUpdateDeferred: CompletableDeferred<Unit>? = null
private var convergenceDeferred: CompletableDeferred<Unit>? = null
suspend fun awaitAeModeUpdate(targetAeMode: Int) {
this.targetAeMode = targetAeMode
aeModeUpdateDeferred = CompletableDeferred()
aeModeUpdateDeferred?.await()
}
suspend fun awaitAeAwbConvergence() {
convergenceDeferred = CompletableDeferred()
convergenceDeferred?.await()
}
private fun process(result: CaptureResult) {
// Checks if AE mode is updated and completes any awaiting Deferred
aeModeUpdateDeferred?.let {
val aeMode = result[CaptureResult.CONTROL_AE_MODE]
if (aeMode == targetAeMode) {
it.complete(Unit)
aeModeUpdateDeferred = null
}
}
// Checks for convergence and completes any awaiting Deferred
convergenceDeferred?.let {
val aeState = result[CaptureResult.CONTROL_AE_STATE]
val awbState = result[CaptureResult.CONTROL_AWB_STATE]
val isAeReady = (
aeState == null
|| aeState == CaptureResult.CONTROL_AE_STATE_CONVERGED
|| aeState == CaptureResult.CONTROL_AE_STATE_FLASH_REQUIRED
)
val isAwbReady = (
awbState == null
|| awbState == CaptureResult.CONTROL_AWB_STATE_CONVERGED
)
if (isAeReady && isAwbReady) {
it.complete(Unit)
convergenceDeferred = null
}
}
}
override fun onCaptureCompleted(
session: CameraCaptureSession,
request: CaptureRequest,
result: TotalCaptureResult
) {
super.onCaptureCompleted(session, request, result)
process(result)
}
}
This callback allows us to do two things: Using awaitAeModeUpdate(targetAeMode: Int) we can specify a specific auto exposure mode we want to wait for. Once it has been reached, the deferred is completed so we can listen to it and wait for the correct state. This can be expanded to not only listen for a specific AE mode — in theory you could wait for any specific condition on the camera. Using awaitAeAwbConvergence() we can wait for auto exposure and white balance to converge. Once they are in a converged state, the deferred is completed and we can continue execution. Now taking a picture is just a matter of orchestrating everything together nicely:
fun takePicture() {
val camera = mCameraDevice ?: return
val session = mCaptureSession ?: return
val builder = mPreviewRequestBuilder ?: return
val imageReader = mImageReader ?: return
/*
To avoid having to have the flash mode enabled all the time during preview,
we can store a reference to the builder and update its repeating request
before performing the precapture sequence.
*/
builder.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_ON_ALWAYS_FLASH)
session.setRepeatingRequest(builder.build(), repeatingCaptureCallback, mBackgroundHandler)
/*
The following is the core of the flash procedure.
It is important that it is performed asynchronously using a different coroutine
scope to allow the main thread to continue updating the preview.
*/
lifecycle.lifecycleScope.launch(Dispatchers.IO) {
/*
We wait to continue until the flash mode has been switched
*/
repeatingCaptureCallback.awaitAeModeUpdate(CaptureRequest.CONTROL_AE_MODE_ON_ALWAYS_FLASH)
runPrecaptureSequence()
val captureRequest =
camera.createCaptureRequest(CameraDevice.TEMPLATE_STILL_CAPTURE).apply {
addTarget(imageReader.surface)
set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_ON_ALWAYS_FLASH)
set(CaptureRequest.FLASH_MODE, CaptureRequest.FLASH_MODE_OFF)
}.build()
session.capture(captureRequest, object : CameraCaptureSession.CaptureCallback() {
override fun onCaptureCompleted(
session: CameraCaptureSession,
request: CaptureRequest,
result: TotalCaptureResult
) {
Log.d(TAG, "Picture taken with flash")
builder.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_ON)
session.setRepeatingRequest(
builder.build(),
repeatingCaptureCallback,
mBackgroundHandler
)
}
}, mBackgroundHandler)
}
}
The first thing to note here is that we are reusing our capture request builder to turn on flash mode. Secondly, it's important to start the whole procedure in a coroutine to avoid blocking the main thread. If we were to launch this on the main thread, the preview would freeze until the picture has been taken. As you can see, we are also utilizing awaitAeModeUpdate() to wait until the flash mode has been enabled. Once that is the case, we run the precapture sequence (below), after which we perform a simple capture() request to take our image. Finally, we reset the auto exposure mode back to default to prevent the flash from being active all the time.
The precapture sequence is what triggers our metering, waits until convergence, and then continues.
private suspend fun runPrecaptureSequence() {
val session = mCaptureSession ?: throw IllegalArgumentException("No capture session")
val previewSurface = mPreviewSurface ?: throw IllegalArgumentException("No preview surface")
val captureRequest = session.device.createCaptureRequest(
CameraDevice.TEMPLATE_PREVIEW
).apply {
addTarget(previewSurface)
set(
CaptureRequest.CONTROL_AE_PRECAPTURE_TRIGGER,
CaptureRequest.CONTROL_AE_PRECAPTURE_TRIGGER_START
)
set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_ON_ALWAYS_FLASH)
set(CaptureRequest.FLASH_MODE, CaptureRequest.FLASH_MODE_OFF)
}
/**
* The [CompletableDeferred] is used to wait for the camera to acknowledge that the
* AE precapture trigger request has been captured/processed. This does NOT mean the
* precapture sequence is complete - it just confirms the trigger was successfully sent.
*/
val precaptureDeferred = CompletableDeferred<Unit>()
session.capture(captureRequest.build(), object : CameraCaptureSession.CaptureCallback() {
override fun onCaptureCompleted(
session: CameraCaptureSession,
request: CaptureRequest,
result: TotalCaptureResult
) {
precaptureDeferred.complete(Unit)
}
}, mBackgroundHandler)
precaptureDeferred.await()
/**
* Now that the precapture trigger has been sent and acknowledged, we wait for the actual
* precapture sequence to complete. This means waiting for AE (auto-exposure) and AWB
* (auto white balance) to converge to stable values before taking the picture.
*/
repeatingCaptureCallback.awaitAeAwbConvergence()
}
That's it. With this you should be able to perform a flash sequence and avoid all the headache of correctly timing all the individual steps.
Conclusions
While the Camera2 API is notoriously difficult to work with, once you get the hang of it the components start to click. Using the modern approach with completable deferreds additionally simplifies synchronization of different tasks and could be applied to many other use cases.
