ハスケルGStreamerティー要素(1-N)のトラブル

私が持っている問題は、次のコードに関連しています:

module Main(main) where

import qualified Media.Streaming.GStreamer as GS
import Data.Maybe
import System.IO
import System.Exit
import System.Glib.MainLoop as Glib
import System.Glib.Signals as Glib
import System.Glib.Properties as Glib


makeElement:: String → String → IO GS.Element
makeElement elementType elementName = do
    element ← GS.elementFactoryMake elementType (Just elementName)
    case element of
        Just element' → return element'
        Nothing → do
            hPutStrLn stdout ("Cannot create element!")
            hFlush stdout
            exitFailure

player =  do
    GS.init

    pipeline ← GS.pipelineNew "video-stream"

    source  ← makeElement "v4l2src" "video-source"
    color   ← makeElement "ffmpegcolorspace" "video-color"
    tee     ← makeElement "tee" "stream-tee"
    rQ      ← makeElement "queue" "record-queue"
    vQ      ← makeElement "queue" "video-queue"
    encoder ← makeElement "y4menc" "video-encoder"
    rSink   ← makeElement "filesink" "record-sink"
    sink    ← makeElement "ximagesink" "video-sink"

    let elements = [source,color,encoder,rSink,vQ,rQ,sink,tee]

    Glib.objectSetPropertyString "location" rSink "rec"

    mapM_ (GS.binAdd (GS.castToBin pipeline)) elements

    -- Request Pads from tee
    dPad ← GS.elementGetRequestPad tee "src%d"
    rPad ← GS.elementGetRequestPad tee "src%d"
    -- Request Static Pads from queue
    sDPad ← GS.elementGetStaticPad vQ "sink"
    sRPad ← GS.elementGetStaticPad rQ "sink"
    -- Link tee source to queue sink
    GS.padLink (fromJust dPad) (fromJust sDPad)
    GS.padLink (fromJust rPad) (fromJust sRPad)

    GS.elementReleaseRequestPad tee $ fromJust dPad
    GS.elementReleaseRequestPad tee $ fromJust rPad

    GS.elementLink source color
    GS.elementLink color tee
    GS.elementLink vQ sink
    GS.elementLink rQ encoder
    GS.elementLink encoder rSink


    GS.elementSetState pipeline GS.StatePlaying

main = do
    loop ← Glib.mainLoopNew Nothing False
    player
    Glib.mainLoopRun loop

コードは、カメラのLEDスイッチをONにしてコンパイルし、ファイルは作成されますが、その後は何も表示されません。 teeとqueue要素がなければ、ビデオの録画/表示のための別個のセットアップはうまくいきます。また、gst-launchでテストすると、同じパイプラインが完全に機能します。 私はgstreamerの仕組みについてここで何かを見逃していますが、何が分からないのですか?

また、役立つならば、私はArchLinuxを使って次のものを使ってビルドしています:
- GHC 7.0.3;
- gstreamer-bindings 0.12.1;
- gtk2hs 0.12.2;
- gstreamer 0.10.35-1;
- glib 1.2.10-9。

9
あなたの発見を答えとして加え、その答えを受け入れるべきです。ここであなた自身の質問に答えるのは失礼だとは思われません。もしあなたが何が間違っていたかを知る最初の人であれば、より多くの力をあなたに!
追加された 著者 Daniel Wagner,

1 答え

解決済み

私は私の解決策を見つけました。それに続くのは長い投稿ですが、私と一緒にいてください。私は誰かと私の不満を分かち合う必要があります。

より多くのバグを試した後、私はgst-launchを使っていくつかの設定をテストすることに戻りました。 これは、私が考えている正しいビデオフォーマットを設定するために別のffmpegcolorspace要素を必要とするfilesinkに行く部分をバッファリングするキュー要素の後に見つけることができました。 この時点で、私はこの事をハスケルからもう一度試してみることに戻っていませんでした。私は「近くに」なる必要があると思ったので、私はCでそれを試しました。 副注釈として、私はCを知らない、構文を理解することができますが、それはそれについてです...そして良さのために、私はちょうど今ハスケルを学ぼうとしています。 私はパッドがキューにリンクすることを確認することができますので、私はまた、GSelementGetCompatiblePad 'ティー要素で使用することを試みることにしました。

一緒に縫い合わされたCコードはこれです:

#include 
#include 

int
main (int argc,char *argv[])
{

    GstElement *pipeline, *source, *color, *color2 , *color3, *tee, *rQ, *vQ, *encoder,   *fSink , *sink;
    GMainLoop *loop;
    loop = g_main_loop_new (NULL,FALSE);
    /* initialize gstreamer */
    gst_init(&argc,&argv);

    /* creating elements */
    pipeline = gst_pipeline_new("stream-pipeline");

    source = gst_element_factory_make ("v4l2src","stream-source");
    color = gst_element_factory_make ("ffmpegcolorspace","video-color");
    tee = gst_element_factory_make ("tee","stream-tee");
    rQ = gst_element_factory_make ("queue","record-queue");
    vQ = gst_element_factory_make ("queue","video-queue");
    encoder = gst_element_factory_make ("theoraenc","video-encoder");
    fSink = gst_element_factory_make ("filesink","record-sink");
    sink = gst_element_factory_make ("ximagesink","video-sink");
    color2 = gst_element_factory_make ("ffmpegcolorspace","video-color2");
    color3 = gst_element_factory_make ("ffmpegcolorspace","video-color3");
    /*check that the elements were created */

    if (!source || !color || !tee || !rQ || !vQ || !encoder || !fSink || !sink){
        g_printerr("One element could not be created!");
        return -1;
    }
    /*set file output location */
    g_object_set(G_OBJECT (fSink),"location","rec",NULL);

    gst_bin_add_many (GST_BIN(pipeline),
                        source,color,color2,color3,tee,rQ,vQ,encoder,fSink,sink,NULL);

    /* get request pads */
    GstPad *dPad, *rPad, *sDPad, *sRPad;

    sDPad = gst_element_get_static_pad(vQ,"sink");
    sRPad = gst_element_get_static_pad(rQ,"sink");
    dPad = gst_element_get_compatible_pad(tee,sDPad,GST_CAPS_ANY);
    rPad = gst_element_get_compatible_pad(tee,sRPad,GST_CAPS_ANY);

    /*link pads*/
    gst_pad_link(dPad,sDPad);
    gst_pad_link(rPad,sRPad);

    /*unref pads */
    gst_object_unref(GST_OBJECT(dPad));
    gst_object_unref(GST_OBJECT(rPad));
    gst_object_unref(GST_OBJECT(sDPad));
    gst_object_unref(GST_OBJECT(sRPad));

    /*link elements */
    gst_element_link(source,tee);
    gst_element_link_many(rQ,color2,encoder,fSink,NULL);
    gst_element_link_many(vQ,color3,sink),NULL;

    /*set the pipeline state to playing */
    gst_element_set_state(pipeline,GST_STATE_PLAYING);

    g_main_loop_run (loop);

    gst_element_set_state(pipeline,GST_STATE_NULL);
    gst_object_unref(GST_OBJECT(pipeline));

    return 0;

}


In order to use 'gst_element_get_compatible_pad' i had to first get static pads from the queue elements so i hand to switch those four related lines. I try it out, and Abracadabra ...oh no, wait ... camera starts, the file is created and a window with the 'video' pops, but a black window that remains black!


No problem i say , run the program with gst-debug-level=5 ( =)) ) yea, right , try reading the whole output.I give up for the moment and i thought maybe it has something to do with the elements in my pipeline not working right together so i code another pipeline in C but this time something more simple just with audio files.
I had the same result so i decided tu debug again, this time with runlevel 3 and i started reading the whole thing,line by line.


Somewhere in there i found this:

stream-tee:src0とrecord-queue:sinkをリンクしようとしています   stream-tee:src0とvideo-queue:sink

をリンクしようとしています


something nasty is happening here


リンクされたストリームtee:src0とvideo-queue:sink、successful   stream-tee:src0とrecord-queue:sinkをリンクしようとしています   
src stream-tee:src0は既にvideo-queue:sink

にリンクされていました。


And it gives up!
I guess i must go back using gst_element_get_request_pad, but haven't i tried that already? So i switch back to vim and replace all occurrences of 'gst_element_get_compatible_pad with the request counterpart like so:

sDPad = gst_element_get_static_pad(vQ,"sink");
sRPad = gst_element_get_static_pad(rQ,"sink");
dPad = gst_element_get_request_pad(tee,"src%d");
rPad = gst_element_get_request_pad(tee,"src%d");


I gaze upon this code and i say to myself 'you twit', this is where it all started; take a deep breath ; after all this is what the debugger complains about so i compile , i run, and Voila. I found my solution.


Those four lines had to be reversed, i had to first get a reference to the static pads and then request a reference to a 'request' pad on the tee element.
I go back to haskell a happy man.I implement my solution, compile, fire up,camera starts, the file is created and ... just like that..nothing, not even the black screen.
Filled with anger i just comment out the lines where i release the request pads and decide to compile and run once more, my neck started to hurt a while ago.
Again, by magic it all works , i have video on the screen and in the file.
I guess Haskell just likes to hold tighter and sometimes you have to just go with something that makes no sense. The gstreamer docs state clearly release,release,release.

最終的なハッスルコード:

module Main(main) where

import qualified Media.Streaming.GStreamer as GS
import Data.Maybe
import System.Exit
import System.Glib.MainLoop as Glib
import System.Glib.Signals as Glib
import System.Glib.Properties as Glib

makeElement:: String → String → IO GS.Element
makeElement elementType elementName = do
        element ← GS.elementFactoryMake elementType (Just elementName)
        case element of
            Just element' → return element'
            Nothing → do
                    putStrLn "Cannot create element!"
                    exitFailure

linkSPadToStaticSink::(GS.ElementClass object, GS.ElementClass elementT) ⇒ object →     elementT → IO (Glib.ConnectId object)
linkSPadToStaticSink elSrc elSink = do
            Glib.on elSrc GS.elementPadAdded (λpad → do
                                                    sinkPad ← GS.elementGetStaticPad elSink "sink"
                                                    GS.padLink pad (fromJust sinkPad)
                                                    return ∅)

player =  do
        GS.init
        pipeline ← GS.pipelineNew "video-stream"
        source ← makeElement "v4l2src" "video-source"
        color ← makeElement "ffmpegcolorspace" "video-color"
        color2 ← makeElement "ffmpegcolorspace" "video-color2"
        tee ← makeElement "tee" "stream-tee"
        rQ ← makeElement "queue" "record-queue"
        vQ ← makeElement "queue" "video-queue"
        encoder ← makeElement "y4menc" "video-encoder"
        rSink ← makeElement "filesink" "record-sink"
        sink ← makeElement "ximagesink" "video-sink"

        let elements = [source,color,color2,encoder,rSink,vQ,rQ,sink,tee]

        Glib.objectSetPropertyString "location" rSink "rec"

        mapM_ (GS.binAdd (GS.castToBin pipeline)) elements

        -- Get static pads from queue elements
        sDPad ← GS.elementGetStaticPad vQ "sink"
        sRPad ← GS.elementGetStaticPad rQ "sink"
        -- Request pads from tee element
        dPad ← GS.elementGetRequestPad tee "src%d"
        rPad ← GS.elementGetRequestPad tee "src%d"
        -- Link tee source to queue sink
        GS.padLink (fromJust dPad) (fromJust sDPad) 
        GS.padLink (fromJust rPad) (fromJust sRPad)

        GS.elementLink source color
        GS.elementLink color tee
        GS.elementLink vQ sink
        GS.elementLink rQ color2
        GS.elementLink color2 encoder
        GS.elementLink encoder rSink

        GS.elementSetState pipeline GS.StatePlaying

main = do
    loop ← Glib.mainLoopNew Nothing False
    player
    Glib.mainLoopRun loop


Now i ask you, should/could i have seen this ?
Was it that obvious ?


I'm glad this will make me be more careful and look in not so obvious places but...eww.

In conclusion to all this, i learned about the gstreamer debug options, i learned that it whispers to me and i MUST listen. I learned about GDB being forced to used because when i began stitching C code all i got was a 'seg fault'.
I learned to love lazy-eval and pure Haskell code.
A little bit of Haskell, maybe a tiny bit of C and more experience. 'Lost' about half a day, three classes and several hours of sleep but after all...So it goes...

10
追加された