Convert CMSampleBuffer to UIImage

by mumu   Last Updated April 19, 2019 12:26 PM

I am trying to convert sampleBuffer to a UIImage and display it in an image view with colorspaceGray. But it displays as the following image. I think there is a problem regarding the conversion. How can I convert the CMSampleBuffer?

It is a gray version of a red colored image

func captureOutput(_ captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, from connection: AVCaptureConnection!) {
    print("buffered")
    let imageBuffer: CVImageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!
    CVPixelBufferLockBaseAddress(imageBuffer, CVPixelBufferLockFlags(rawValue: 0))
    let width: Int = CVPixelBufferGetWidth(imageBuffer)
    let height: Int = CVPixelBufferGetHeight(imageBuffer)
    let bytesPerRow: Int = CVPixelBufferGetBytesPerRow(imageBuffer)
    let lumaBuffer = CVPixelBufferGetBaseAddress(imageBuffer)

    //let planeCount : Int = CVPixelBufferGetPlaneCount(imageBuffer)
    let grayColorSpace: CGColorSpace = CGColorSpaceCreateDeviceGray()
    let context: CGContext = CGContext(data: lumaBuffer, width: width, height: height, bitsPerComponent: 8, bytesPerRow: bytesPerRow , space: grayColorSpace, bitmapInfo: CGImageAlphaInfo.none.rawValue)!
    let dstImageFilter: CGImage = context.makeImage()!
    let imageRect : CGRect = CGRect(x: 0, y: 0, width: width, height: height)
    context.draw(dstImageFilter, in: imageRect)
    let image = UIImage(cgImage: dstImageFilter)
    DispatchQueue.main.sync(execute: {() -> Void in
        self.imageTest.image = image
    })
}


Answers 2


Looks like the CMSampleBuffer is giving you RGBA data from which you then directly construct the grayscale image. You will either need to construct a new buffer where for each pixel you do something like gray = (pixel.red+pixel.green+pixel.blue)/3. Or you need to create a normal RGBA image from the data you received and then convert it to grayscale.

But in your code you do no transition at all. You took the raw pointer to buffer using CVPixelBufferGetBaseAddress regardless of what sort of data is in there. And then you just pass the same pointer in creating the image which assumes the data received are grayscale.

Matic Oblak
Matic Oblak
March 24, 2017 10:56 AM

The conversion is simple :

func captureOutput(_ captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, from connection: AVCaptureConnection!) {

     let imageBuffer: CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!
     let ciimage : CIImage = CIImage(cvPixelBuffer: imageBuffer)   
     let image : UIImage = self.convert(cmage: ciimage) 
}

// Convert CIImage to CGImage
func convert(cmage:CIImage) -> UIImage
{
     let context:CIContext = CIContext.init(options: nil)
     let cgImage:CGImage = context.createCGImage(cmage, from: cmage.extent)!
     let image:UIImage = UIImage.init(cgImage: cgImage)
     return image
}
mumu
mumu
April 18, 2017 11:17 AM

Related Questions


Updated April 02, 2017 21:26 PM

Updated June 29, 2017 23:26 PM

Updated June 07, 2019 07:26 AM

Updated April 07, 2017 04:26 AM