English 中文(简体)
Live Streaming ( Socket ) - How to sync audio and video?
原标题:

1- Which one is better to use for streaming video ? TCP or UDP socket and why?

2- While streaming live, audio and video are coming from the server separately, so how can i make sure that the video i display and the audio I play on the device are in sync?

最佳回答

I wrote a voice chat application a while ago and TCP was out of the question, UDP multicasting is really the only way to go if you re looking for near-realtime data flow. There s two main issues with streaming stuff over UDP though:

  1. The dropped packets. In the case of audio, it s a pretty easy fix. Usually the dropped packets won t make an audible difference (the packets are decompressed individually). However, when dealing with video, especially if the video is compressed (it usually is), figuring out a proper transfer protocol that ensures network robustness is a daunting task to say the least, especially if you re doing this from scratch. Video frames are split up in various packets. Figuring out what to do when these packets are missing is tough.
  2. Synchronization between audio and video. This is a very tough problem and I suggest reading up on protocols such as RTSP (Real-Time Streaming Protocol). This is not an easy task, but here s some introductory info: http://www.cs.columbia.edu/~hgs/rtsp/ - sometimes it s done by sending separate sync packets (some protocols send these over TCP) that tell the player how the sound should match up with the video.
问题回答

I would do UDP. However it depends on what you want. UDP will drop packets rather than wait (TCP). The trade off is whether you want a stable, but sometimes slow and costly, or one that is efficient, but sometimes may not get delivered. The choice is yours when it comes to how you want to implement it and how you are using it.

Today even youtube streams over HTTP ... here is a nodejs app which streams a file to the browser client ... use as a starting point to live stream video with audio nicely in sync

// usage 
// do following on server side (your laptop running nodejs)
// node this_file.js
//
// then once above is running point your browser at
//    http://localhost:8888
//
// of course your browser could be on your mobile or own custom app



var http = require( http ),
    fs = require( fs ),
    util = require( util );

var path = "/path/to/audio/or/video/file/local/to/server/cool.mp4"; // put any audio or video file here

var port = 8888;
var host = "localhost";

http.createServer(function (req, res) {

  var stat = fs.statSync(path);
  var total = stat.size;

  if (req.headers.range) {   // meaning client (browser) has moved the forward/back slider
                                         // which has sent this request back to this server logic ... cool
    var range = req.headers.range;
    var parts = range.replace(/bytes=/, "").split("-");
    var partialstart = parts[0];
    var partialend = parts[1];

    var start = parseInt(partialstart, 10);
    var end = partialend ? parseInt(partialend, 10) : total-1;
    var chunksize = (end-start)+1;
    console.log( RANGE:   + start +   -   + end +   =   + chunksize);

    var file = fs.createReadStream(path, {start: start, end: end});
    res.writeHead(206, {  Content-Range :  bytes   + start +  -  + end +  /  + total,  Accept-Ranges :  bytes ,  Content-Length : chunksize,  Content-Type :  video/mp4  });
    file.pipe(res);

  } else {

    console.log( ALL:   + total);
    res.writeHead(200, {  Content-Length : total,  Content-Type :  video/mp4  });
    fs.createReadStream(path).pipe(res);
  }
}).listen(port, host);

console.log("Server running at http://" + host + ":" + port + "/");




相关问题
Multi-bitrate/dynamic rate for progressive FLV playback

Is there a way to get multi-bitrate playback working with progessive playback without having to download all versions of the file? People all coming up with all kinds of cool ways to make progressive ...

How to stream large Files using JAXB Marshaller?

The Problem I m facing is how to marshall a large list of objects into a single XML File, so large I can not marshall the complete list in one step. I have a method that returns these objects in ...

将音频Clips从Peter改为服务器

我不禁要问,那里是否有任何实例表明从Peit向服务器发送音响。 I m不关心电话或SIP风格的解决办法,只是一个简单的袖珍流程......

热门标签