Can LTMM do this?

Hi.

I have a very specific need for which I am having a very difficult time finding a library that will meet the requirements. I'd like to know if LEADTOOLs is capable of handling the following scenario from end to end:<br />
<br />
A server application must be able of maintaining a constant MPEG-2 TS stream to a client. The stream's source must be either a video with audio or a bitmap image which can be updated in real time. The video must be demuxed and decoded on the server so that an overlay filter may be inserted. The output must then be reencoded and muxed for MPEG-2 TS streaming to the client. I am using .NET 3.5.<br />
<br />
The LEADTOOLS documentation is not sufficient to learn from and examples are a poor way to document an API as complex as the MM SDK. They are geared for the copy and paste programmer: "I don't know what this line does but I know it has to be there otherwise it doesn't work." I'm very frustrated with LEADTOOLS right now.

The application you are attempting to build is possible to create with the LEADTOOLS Multimedia Toolkit.

I would suggest starting with our Restreamer demo which will allow you to choose a file or files and stream them over MPEG2 Transport. Unfortunately, this demo is currently only available in C++. For .NET implementation, what you want to do is:
- implement a convert control
- set the targetformat to MPEG2 Transport
- set the targetfile to udp://
- insert any filters
- set the sourcefile (video or bitmap)
- call startconvert

With respect to receiving, you'll want to look over our MPEG2 Transport Demo (available in .NET) which will allow you to connect to the server. You could insert the filter here as well, if you wanted to do so with the player control.

We are constantly working to improve our documentation. We make all improvements available as fast as we can. We are working to include more .NET samples as well as demos. If you have any questions regarding our products or the use of them, we encourage you to email us and use our forums for technical support.

Thanks for your help, Walter. I have been able to adapt a .NET implementation of the Restreamer application you pointed out to me.

I have a new question. How do I construct a bitmap image from the data provided to ILMVUserCallback.ReceiveProc? The arguments passed to ReceiveProc do not contain the color component ordering (RGBA or ARGB?) and there is no no indication of the stride.

Thanks

Another issue:

After adding my callback filter, the callback filter's ReceiveProc method is never invoked.

Here's my code to add the filter:

public static void SetCallbackFilter(LMVCallbackLib.ILMVUserCallback userCallback)
{
int index = _convert.VideoProcessors.Find(
"@device:sw:{E526D606-22E7-494C-B81E-AC0A94BFE603}\\" +
"{E2B7DDB0-38C5-11D5-91F6-00104BDB8FF9}");

_convert.SelectedVideoProcessors.Add(_convert.VideoProcessors.Item(index), -1);
var callback = (LMVCallbackLib.ILMVCallback)_convert.GetSubObject(
(int)LTMMLib.ltmmConvert_Object.ltmmConvert_Object_SelVideoProcessor);
callback.ReceiveProcObj = userCallback;
return;
}

And here's my callback class:

internal class ImageCallback : LMVCallbackLib.ILMVUserCallback
{
public void ReceiveProc(int data, int width, int height, int bitCount, int byteCount, int isTopDown)
{
Console.WriteLine("Callback {0}, {1}, {2}, {3}", width, height, bitCount, byteCount);
}
}

The Console.WriteLine is never performed.

I will write a small demo to try and illustrate what you are looking for. I'll post it as soon as it's ready.

This demo takes the input video and flips it upside-down.

Thanks for the demo, Walter. It still seems that I cannot get the callback filter to be invoked in my application. From what I can tell, I have been following the same steps you have. The only difference I can see is that I am using the convert class instead of the playback class and my index argument was -1 for SelectedVideoProcessors.Add. I have changed it to 0, but that didn't help.

Does the convert class support filters?

The convert class should be able to take any Direct show filter just like the playback class. Try looping through the avalable processors and check each for the friendly name. Insert this code into the demo I've provided.

for(int i=0; i < axltmmPlayCtrl1.VideoProcessors.Count; i++)
{
ltmmProcessor processor = axltmmPlayCtrl1.VideoProcessors.Item(i);
Console.WriteLine(processor.FriendlyName.ToString());
}

Once you see the list, you can attempt to select the processor using the friendly name. Replace this line:
nIndex = axltmmPlayCtrl1.VideoProcessors.Find(videoProcessorName);

With this:

int i = 0;
bool foundCallback = false;
while (i < axltmmPlayCtrl1.VideoProcessors.Count && !foundCallback)
{
ltmmProcessor processor = axltmmPlayCtrl1.VideoProcessors.Item(i);
Console.WriteLine(processor.FriendlyName.ToString());
if (processor.FriendlyName.ToString().Equals("LEAD Video Callback Filter (2.0)"))
{
Console.WriteLine("Callback filter found");
foundCallback = true;
nIndex = i;
}
i++;
}

This is the original way that I was doing it as per the examples provided with LEADTools. I thought it was a little kludgy so I used the Find method, which is the right tool for the job. Neither way worked for me anyway. When I step through the example you've written, the processor index is 36 which matches the processor index it finds in my application. I have already gone as far as to write out the names of the filters. The filter name reported by .FriendlyName is "LEAD Video Callback Filter (2.0)".

Here's the project (if it attaches).

This is a modified demo which includes a picturebox. It is for your questions about the stride.

Thanks again, Walter. I appreciate all of your assistance.

I thought this is how I would convert to a Bitmap, but I wasn't sure whether I could count on stride being a direct function of the bits per pixel value and width. I wasn't aware of a rule dictating that must be true for a standard windows bitmap, though I did know that they must be four byte aligned.

As for the "no callback using the convert class" issues I'm experiencing... I cannot make it work. I am certain I am doing it exactly as you've described that I should...

Any word on the callback filter and capture class? I am sorry about being a pest over this, but getting this one thing working is the only thing standing in my way of getting budget approval to purchase the LEAD toolkits I hope to use.

I appologize for the delay.

The main problem is the callback filter needs to be in place before the StartConvert method is called on the convert control.

After some adjustments, I was able to get your project working.

Great! Thank you very much. That was pretty simple, really...

Walter, unfortunately when changing the TargetFormat from AVI back to MPEG-2 TS the example you provided does not work. Instead it hangs for approximately five seconds when StartConvert() is invoked, then nothing happens. The callback filter never invokes the ReceiveProc method on my object. A requirement of my application is that it is compatible with MPEG-2 decoder cards because of our hardware.

I would like to add that when changing TargetFormat to MPEG-2, I am also changing the TargetFile appropriately.

I doubt the TargetFile should have any affect, it's just a filename.

I believe the problem was occuring because when you set the destination format as AVI, it must decompress the file to convert it.  Since the source file is MPEG2 Transport, when you set the destination format as MPEG2Transport, it's not going to try decompressing it unless you specifically tell it to by giving it audio and video decompressors.

I modified your StartStreaming method like this and it no longer locked up after StartConvert and everything appeared to work as expected.

public static void StartStreaming(ltmmConvert convert, string target, string source)
{
    convert.TargetFormat = (int)ltmmConvert_TargetFormat.ltmmConvert_TargetFormat_MPEG2_TRANSPORT;
    //convert.TargetFormat = (int)ltmmConvert_TargetFormat.ltmmConvert_TargetFormat_Avi;
    convert.TargetFile = target;
    convert.sourcefile = source;

    // Set video compressor to MPEG2 encoder
    int index = -1;
    for (int i = 0; i < convert.VideoCompressors.Count; i++)
    {
        if (convert.VideoCompressors.Item(i).FriendlyName == "LEAD MPEG2 Encoder (3.0)")
            index = i;
    }
    convert.VideoCompressors.Selection = index;

    // Set audio compressor to MPEG encoder
    index = -1;
    for (int i = 0; i < convert.AudioCompressors.Count; i++)
    {
        if (convert.AudioCompressors.Item(i).FriendlyName == "LEAD MPEG Audio Encoder (2.0)")
            index = i;
    }
    convert.AudioCompressors.Selection = index;

    convert.StartConvert();
}

Thanks very much! I wasn't aware that the encoders needed to be manually added because targetformat was set to MPEG2TS, though it does now make sense. Thank you both, Walter and Greg.