C# Camera API for Getting Video Frame

If you have a USB camera, how can you build a simple C# camera application on Windows 10? There are three options:  WIA (Windows Imaging Acquisition), DirectShow and MediaCapture. After trying some sample code that downloaded from CodeProject and GitHub, I got the conclusion: 1. WIA is not good because it does not support my webcam.  2. DirectShow can work well, but there is no C# API provided by Microsoft. You need to create a wrapper for C++ API. 3. MediaCapture class that designed for UWP apps provides C# APIs which provide low-level control over the capture pipeline and enable advanced capture scenarios. In this article, I want to share how to create a simple C# webcam app in which I can handle every preview frame myself.

UWP media capture camera preview

Getting Preview Frames via C# Camera API

Although the reference page of MediaCapture class is informative, to quickly learn APIs, I prefer to get started with UWP sample code that provided by Microsoft. Press Ctrl+F to search for camera, you will see following samples:

UWP camera samples

If you want to add custom effects or functionalities (OCR, barcode etc.) for the camera, a common way is to register a callback function for receiving every preview frame. After running all samples, I found the project CameraFrames is what I want.

How to build a UWP webcam app in C#

  1. To use webcam, double-click package.appxmanifest to check the corresponding capability:
    UWP webcam capability
  2. Create an image element for rendering preview frames:
    UWP webcam preview image
  3. Initialize FrameRenderer with the image element:
    _frameRenderer = new FrameRenderer(PreviewImage);
  4. Initialize the media capture object:
                    // Create a new media capture object.
                    _mediaCapture = new MediaCapture();
    
                    var settings = new MediaCaptureInitializationSettings()
                    {
                        // Select the source we will be reading from.
                        SourceGroup = groupModel.SourceGroup,
    
                        // This media capture has exclusive control of the source.
                        SharingMode = MediaCaptureSharingMode.ExclusiveControl,
    
                        // Set to CPU to ensure frames always contain CPU SoftwareBitmap images,
                        // instead of preferring GPU D3DSurface images.
                        MemoryPreference = MediaCaptureMemoryPreference.Cpu,
    
                        // Capture only video. Audio device will not be initialized.
                        StreamingCaptureMode = StreamingCaptureMode.Video,
                    };
    
                    try
                    {
                        // Initialize MediaCapture with the specified group.
                        // This can raise an exception if the source no longer exists,
                        // or if the source could not be initialized.
                        await _mediaCapture.InitializeAsync(settings);
                        _logger.Log($"Successfully initialized MediaCapture for {groupModel.DisplayName}");
                    }
                    catch (Exception exception)
                    {
                        _logger.Log(exception.Message);
                        DisposeMediaCapture();
                    }
  5. Use DeviceWatcher to list all connected devices:
            var deviceSelector = MediaFrameSourceGroup.GetDeviceSelector();
            _watcher = DeviceInformation.CreateWatcher(deviceSelector);
            _watcher.Added += Watcher_Added;
            _watcher.Removed += Watcher_Removed;
            _watcher.Updated += Watcher_Updated;
            _watcher.Start();
    
            private async void Watcher_Added(DeviceWatcher sender, DeviceInformation args)
            {
                await AddDeviceAsync(args.Id);
            }
    
            private async Task AddDeviceAsync(string id)
            {
                var group = await MediaFrameSourceGroup.FromIdAsync(id);
                if (group != null)
                {
                    await _dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
                        {
                            _sourceCollection.Add(new FrameSourceGroupModel(group));
                        });
                }
            }
    
  6. Updates the current frame source to the one corresponding to the user’s selection:
    _mediaCapture.FrameSources.TryGetValue(info.SourceInfo.Id, out _source);
  7. Initialize MediaFrameReader and register a callback function:
                if (_source != null)
                {
                    _reader = await _mediaCapture.CreateFrameReaderAsync(_source);
                    _reader.FrameArrived += Reader_FrameArrived;
                }
  8. Start reading webcam frames:
    MediaFrameReaderStartStatus result = await _reader.StartAsync();
  9. Receive all frames and render them on image element:
            private void Reader_FrameArrived(MediaFrameReader sender, MediaFrameArrivedEventArgs args)
            {
                // TryAcquireLatestFrame will return the latest frame that has not yet been acquired.
                // This can return null if there is no such frame, or if the reader is not in the
                // "Started" state. The latter can occur if a FrameArrived event was in flight
                // when the reader was stopped.
                using (var frame = sender.TryAcquireLatestFrame())
                {
                    _frameRenderer.ProcessFrame(frame);
                }
            }
    
            public void ProcessFrame(MediaFrameReference frame)
            {
                var softwareBitmap = FrameRenderer.ConvertToDisplayableImage(frame?.VideoMediaFrame);
    
                if (softwareBitmap != null)
                {
                    // Swap the processed frame to _backBuffer and trigger UI thread to render it
                    softwareBitmap = Interlocked.Exchange(ref _backBuffer, softwareBitmap);
    
                    // UI thread always reset _backBuffer before using it.  Unused bitmap should be disposed.
                    softwareBitmap?.Dispose();
    
                    // Changes to xaml ImageElement must happen in UI thread through Dispatcher
                    var task = _imageElement.Dispatcher.RunAsync(CoreDispatcherPriority.Normal,
                        async () =>
                        {
                            // Don't let two copies of this task run at the same time.
                            if (_taskRunning)
                            {
                                return;
                            }
                            _taskRunning = true;
    
                            // Keep draining frames from the backbuffer until the backbuffer is empty.
                            SoftwareBitmap latestBitmap;
                            while ((latestBitmap = Interlocked.Exchange(ref _backBuffer, null)) != null)
                            {
                                var imageSource = (SoftwareBitmapSource)_imageElement.Source;
                                await imageSource.SetBitmapAsync(latestBitmap);
                                latestBitmap.Dispose();
                            }
    
                            _taskRunning = false;
                        });
                }
            }

Reference

Basic photo, video, and audio capture with MediaCapture

Source Code

https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/CameraFrames