English 中文(简体)
Microsoft Media Foundation - Decode h264 samples
原标题:

I used the Microsoft Media Foundation samples (namely MFCaptureToFile) to capture H264 frames from my webcam and write them to a file.

I m trying to use IMFTransform to decode the captured frames and get the underline images out (YUV, BMP, whatever).

However, the ProcessInput method never returns with MF_E_NOTACCEPTING, and the ProcessOutput method always return MF_E_TRANSFORM_NEED_MORE_INPUT.

I basically read every frame and call ProcessInput on it.

Any ideas? Can someone modify the MFCaptureToFile sample to show me how it is done? I m doing all my processing under CCapture::OnReadSample.

Any help would be greatly appreciated!

Ido

问题回答

I have been able to successfully use the MF H264 decoder MFT to deocde frames stored in an .mp4 file to raw YUV. The full code sample is available here.

The critical pieces are creating the H264 decoder MFT and then supplying it with samples. I ve included the snippets of code for those two bits below.

// Create H.264 decoder.
CHECK_HR(CoCreateInstance(CLSID_CMSH264DecoderMFT, NULL, CLSCTX_INPROC_SERVER,
    IID_IUnknown, (void**)&spDecTransformUnk), "Failed to create H264 decoder MFT.
");

CHECK_HR(spDecTransformUnk->QueryInterface(IID_PPV_ARGS(&pDecoderTransform)), "Failed to get IMFTransform interface from H264 decoder MFT object.
");

MFCreateMediaType(&pDecInputMediaType);
CHECK_HR(pFileVideoMediaType->CopyAllItems(pDecInputMediaType), "Error copying media type attributes to decoder input media type.
");
CHECK_HR(pDecoderTransform->SetInputType(0, pDecInputMediaType, 0), "Failed to set input media type on H.264 decoder MFT.
");

MFCreateMediaType(&pDecOutputMediaType);
pDecOutputMediaType->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video);
pDecOutputMediaType->SetGUID(MF_MT_SUBTYPE, MFVideoFormat_IYUV);
CHECK_HR(MFSetAttributeSize(pDecOutputMediaType, MF_MT_FRAME_SIZE, VIDEO_SAMPLE_WIDTH, VIDEO_SAMPLE_HEIGHT), "Failed to set frame size on H264 MFT out type.
");
CHECK_HR(MFSetAttributeRatio(pDecOutputMediaType, MF_MT_FRAME_RATE, 30, 1), "Failed to set frame rate on H264 MFT out type.
");
CHECK_HR(MFSetAttributeRatio(pDecOutputMediaType, MF_MT_PIXEL_ASPECT_RATIO, 1, 1), "Failed to set aspect ratio on H264 MFT out type.
");
pDecOutputMediaType->SetUINT32(MF_MT_INTERLACE_MODE, 2);

CHECK_HR(pDecoderTransform->SetOutputType(0, pDecOutputMediaType, 0), "Failed to set output media type on H.264 decoder MFT.
");

CHECK_HR(pDecoderTransform->GetInputStatus(0, &mftStatus), "Failed to get input status from H.264 decoder MFT.
");
if (MFT_INPUT_STATUS_ACCEPT_DATA != mftStatus) {
    printf("H.264 decoder MFT is not accepting data.
");
    goto done;
}

CHECK_HR(pDecoderTransform->ProcessMessage(MFT_MESSAGE_COMMAND_FLUSH, NULL), "Failed to process FLUSH command on H.264 decoder MFT.
");
CHECK_HR(pDecoderTransform->ProcessMessage(MFT_MESSAGE_NOTIFY_BEGIN_STREAMING, NULL), "Failed to process BEGIN_STREAMING command on H.264 decoder MFT.
");
CHECK_HR(pDecoderTransform->ProcessMessage(MFT_MESSAGE_NOTIFY_START_OF_STREAM, NULL), "Failed to process START_OF_STREAM command on H.264 decoder MFT.
")

Once the decoder has been created and you are getting the encoded H264 frames from somewhere you need to pass them to the MFT created above.

    MFCreateSample(&reConstructedVideoSample);
CHECK_HR(MFCreateMemoryBuffer(srcBufLength, &reConstructedBuffer), "Failed to create memory buffer.
");
CHECK_HR(reConstructedVideoSample->AddBuffer(reConstructedBuffer), "Failed to add buffer to re-constructed sample.
");
CHECK_HR(reConstructedVideoSample->SetSampleTime(llVideoTimeStamp), "Error setting the recon video sample time.
");
CHECK_HR(reConstructedVideoSample->SetSampleDuration(llSampleDuration), "Error setting recon video sample duration.
");

byte *reconByteBuffer;
DWORD reconBuffCurrLen = 0;
DWORD reconBuffMaxLen = 0;
CHECK_HR(reConstructedBuffer->Lock(&reconByteBuffer, &reconBuffMaxLen, &reconBuffCurrLen), "Error locking recon buffer.
");
memcpy(reconByteBuffer, srcByteBuffer, srcBuffCurrLen);
CHECK_HR(reConstructedBuffer->Unlock(), "Error unlocking recon buffer.
");
reConstructedBuffer->SetCurrentLength(srcBuffCurrLen);

CHECK_HR(srcBuf->Unlock(), "Error unlocking source buffer.
");

CHECK_HR(pDecoderTransform->ProcessInput(0, reConstructedVideoSample, 0), "The H264 decoder ProcessInput call failed.
");

CHECK_HR(pDecoderTransform->GetOutputStatus(&mftOutFlags), "H264 MFT GetOutputStatus failed.
");

//if (mftOutFlags == MFT_OUTPUT_STATUS_SAMPLE_READY)
//{
    CHECK_HR(pDecoderTransform->GetOutputStreamInfo(0, &StreamInfo), "Failed to get output stream info from H264 MFT.
");

    while (true)
    {
        CHECK_HR(MFCreateSample(&mftOutSample), "Failed to create MF sample.
");
        CHECK_HR(MFCreateMemoryBuffer(StreamInfo.cbSize, &pBuffer), "Failed to create memory buffer.
");
        CHECK_HR(mftOutSample->AddBuffer(pBuffer), "Failed to add sample to buffer.
");
        outputDataBuffer.dwStreamID = 0;
        outputDataBuffer.dwStatus = 0;
        outputDataBuffer.pEvents = NULL;
        outputDataBuffer.pSample = mftOutSample;

        mftProcessOutput = pDecoderTransform->ProcessOutput(0, 1, &outputDataBuffer, &processOutputStatus);

        if (mftProcessOutput != MF_E_TRANSFORM_NEED_MORE_INPUT)
        {
            // ToDo: These two lines are not right. Need to work out where to get timestamp and duration from the H264 decoder MFT.
            CHECK_HR(outputDataBuffer.pSample->SetSampleTime(llVideoTimeStamp), "Error getting YUV sample time.
");
            CHECK_HR(outputDataBuffer.pSample->SetSampleDuration(llSampleDuration), "Error getting YUV sample duration.
");

            IMFMediaBuffer *buf = NULL;
            DWORD bufLength;
            CHECK_HR(mftOutSample->ConvertToContiguousBuffer(&buf), "ConvertToContiguousBuffer failed.
");
            CHECK_HR(buf->GetCurrentLength(&bufLength), "Get buffer length failed.
");

            printf("Writing sample %i, sample time %I64d, sample duration %I64d, sample size %i.
", sampleCount, yuvVideoTimeStamp, yuvSampleDuration, bufLength);

            byte *byteBuffer;
            DWORD buffCurrLen = 0;
            DWORD buffMaxLen = 0;
            buf->Lock(&byteBuffer, &buffMaxLen, &buffCurrLen);
            outputBuffer.write((char *)byteBuffer, bufLength);
            outputBuffer.flush();
        }
        else {
            break;
        }

        mftOutSample->Release();
   }

The Microsoft H264 Decoder MFT is somewhat special. It internally buffers a lot of samples. (Thats the reason why its not usable for live scenarios as it always introduces a delay of about a second, due to this internal buffering). i guess you have to feed it at least with a full GOP to receive some output samples. Give it a try





相关问题
Transparency of subtitles in Vista / Windows 7

I implemented the EVR renderer into a player of mine to deal with bad resizing quality on Windows Vista+ and came to problems... I have subtitle overlay problems with the EVR: try to see what i m ...

decode MPEG1/2 video with Media Foundation

I am using Media Foundation to play videos. On windows 7 some videos encoded with Mpeg1/Mpeg2 PS and Windows Media Player can play them, but Media Foundation does not(I tried to register a stub ...

Simulating a webcam stream

Is it possible to simulate a web-cam output so that other programs that use web-cams (for example Skype) will recognize it as a real web-cam?

Is there a macro to detect the operating system...?

I am planning to write a piece of code using the media foundation API which is available after vista. I want to add the code inside a #if block something like... #if <SomeMacro> // all the ...

Microsoft Media Foundation - Decode h264 samples

I used the Microsoft Media Foundation samples (namely MFCaptureToFile) to capture H264 frames from my webcam and write them to a file. I m trying to use IMFTransform to decode the captured frames and ...

System.AccessViolationException from unmanaged code?

I m writing this library that implements some basic audio player features in C++/CLI via the Media Foundation framework that will be consumed by managed code. I can play audio, stop, pause, etc just ...

直接使用 直接Show外的放映过滤器?

I m目前与Windows Media Foundation打交道。 然而,由于Microsoft H.264 decoder和一些海关格式缺失的编码存在一些问题,我很想知道这是否是......。

Media Foundation TypeLibrary for managed code?

Is anyone know how to generate media foundation TypeLibrary which will then be imported as COM in managed code ? I didn t found any dll or tlb file related to media foundation which i can import in my ...

热门标签