Screen Capture API Download Multitrack Video Tutorial

Posted on by
Screen Capture API Download Multitrack Video Tutorial

Screen Capture API Download Multitrack Video Tutorial

We were inspired, stumbling upon the great Screen Recorder: recording microphone and the desktop audio at the same time webpage, leading us to think that we could improve the scope of functionality of the recent Screen Capture API Download Audio Video Tutorial where we left off …

  • being able to separately, but not synchronously, setting in play, a video and an audio stream separately playable and “kludgily synchronizable” via a timer mechanism to start the audio after the video … with, today …
  • the screen-record-voice glitch.com inspired, thanks, ability to synchronously record the video along with the audio, and be able to download them, optionally, into a single webm video with a video track and an optional audio track

As far as user interaction goes, we only add one new checkbox element as per …


<input title='Leave checked and capture of button to left (which should be clicked first for any screen capturing) is a single webm video with audio and video tracks synchronized whereas unchecking this checkbox and button to right usage has timer synchronization of a video (with only video tracks) with an audio (only with audio tracks) element playing so as to finish at the same time.' type=checkbox id=caudio checked></input>

… to offer several more combinations of functionalities now, with a single web video containing video and audio a step forward in sharing and collaboration functionalities, with the changed screen_capture_api_va.htm Screen Capture API using video and audio downloadable content (including single video with audio and video track option) web application for you to try below, maybe?

Previous relevant Screen Capture API Download Audio Video Tutorial is shown below.

Screen Capture API Download Audio Video Tutorial

Screen Capture API Download Audio Video Tutorial

The recent Screen Capture API Download Video Tutorial …

  • Screen Capture API inspired “streamed video” screen capturing logic … benefitted from …
  • download capabilities via MediaRecorder “video” functionality … and, today, we add into the mix …
  • recording and download of “audio” content via the device’s “microphone” (permissions granted, that is)

This logic we thought required its own independent audio_idea.html “proof of Audio only MediaRecorder usage concept” web application, the reason being that very little in the “media” wooooorrrrlllllddd passes muster these days if it does not get preceded by a real user “onclick” event induced click of a relevant HTML element (usually a button), and this infers a need to synchronize, and have Javascript variable names that do not overlap many of the “video” ones.


<html>
<head>
<style>
body {
padding: 0;
margin: 0;
}

#amytable {
margin-top: 0px;
margin-left: 0px;
}

td {
vertical-align: top;
}

button {
vertical-align: top;
}
</style>
</head>
<body>
<table id=amytable><tr><td><button id=astart>Audio &#9654;</button>&nbsp;&nbsp;&nbsp;<button onclick="adownload();" id=adownload>Download &#127908;</button>&nbsp;&nbsp;&nbsp;<button id=astop>Stop &#9209;</button><br><br>
<audio id=audio autoplay></audio>

<!--strong id=stronge>Log:</strong>
<br>
<pre id="log"></pre--></td><td style="width:20%;">
<!--h2>Using Screen Capture API</h2><h3>RJM Programming - June, 2022</h3><h4>Thanks to <a target=_blank href='//developer.mozilla.org/en-US/docs/Web/API/Screen_Capture_API/Using_Screen_Capture' title='https://developer.mozilla.org/en-US/docs/Web/API/Screen_Capture_API/Using_Screen_Capture'>https://developer.mozilla.org/en-US/docs/Web/API/Screen_Capture_API/Using_Screen_Capture</a> and <a target=_blank title='https://developer.mozilla.org/en-US/docs/Web/API/MediaStream_Recording_API' href='//developer.mozilla.org/en-US/docs/Web/API/MediaStream_Recording_API'>https://developer.mozilla.org/en-US/docs/Web/API/MediaStream_Recording_API&nbsp;&nbsp;</a></h4><br><br><strong id=stronge>Log:</strong-->
<br>
<pre id="alog"></pre><canvas id=acanvas></canvas></td></tr></table>

<script type='text/javascript'>
var audioElem=document.getElementById('audio');
var amediaRecorder=null, xblob=null, chunks=[], xaudio=null, xaudioURL=null, astream=null, microphone=null, audioCtx=null, acanvas=null;
const astartElem = document.getElementById("astart");
const astopElem = document.getElementById("astop");

//console.log = msg => logElem.innerHTML += `${msg}<br>`;
//console.error = msg => logElem.innerHTML += `<span class="error">${msg}</span><br>`;
//console.warn = msg => logElem.innerHTML += `<span class="warn">${msg}<span><br>`;
//console.info = msg => logElem.innerHTML += `<span class="info">${msg}</span><br>`;

// Set event listeners for the start and stop buttons ... thanks to https://developer.mozilla.org/en-US/docs/Web/API/Screen_Capture_API/Using_Screen_Capture
astartElem.addEventListener("click", function(evt) {
astartCapture();
}, false);

astopElem.addEventListener("click", function(evt) {
astopCapture();
}, false);

async function astartCapture() {
audioCtx = new AudioContext();
if (navigator.mediaDevices) {

//console.log('precanvas');
acanvas = document.querySelector("acanvas");
//console.log('canvas=' + canvas);

// Optional frames per second argument.
//const stream = canvas.captureStream(25);

//console.log(stream);
//window.stream = stream;

navigator.mediaDevices.getUserMedia({"audio": true}).then((stream) => {
audioElem.srcObject = stream;
microphone = audioCtx.createMediaStreamSource(stream);

// Instantiate the media recorder.
amediaRecorder = new MediaRecorder(audioElem.srcObject); //audioElem.srcObject);

// Create a buffer to store the incoming data.
chunks = [];
amediaRecorder.ondataavailable = (event) => {
chunks.push(event.data);
}

amediaRecorder.start(1000);

amediaRecorder.onstop = () => {
// A "blob" combines all the audio chunks into a single entity
xblob = new Blob(chunks, {"type": "audio/ogg; codecs=opus"});
chunks = []; // clear buffer

// One of many ways to use the blob
xaudio = new Audio();
xaudioURL = window.URL.createObjectURL(xblob);
xaudio.src = xaudioURL;
}
// `microphone` can now act like any other AudioNode
}).catch((err) => {
// browser unable to access microphone
// (check to see if microphone is attached)
});
} else {
// browser unable to access media devices
// (update your browser)
}
}

function astopCapture() {
let atracks = audioElem.srcObject.getTracks();

//tracks.forEach(track => amediaRecorder.addTrack(track, stream));
atracks.forEach(track => track.stop());
audioElem.srcObject = null;
if (amediaRecorder) {
amediaRecorder.stop();
}
}

function adownload() {
//console.log("downloading 0");
// let dtracks = videoElem.srcObject.getTracks();
//tracks.forEach(track => mediaRecorder.addTrack(track, stream));
// dtracks.forEach(track => window.stream.addTrack(track));
//console.log("recording1 vs " + mediaRecorder.state);
//mediaRecorder.stop();
//console.log("recording2 vs " + mediaRecorder.state);
if (amediaRecorder) {
if (('' + amediaRecorder.state) != 'inactive') {
amediaRecorder.requestData();
}
//console.log("recording3 vs " + amediaRecorder.state);
xblob = new Blob(chunks, {
type: "audio/ogg; codecs=opus"
});
url = URL.createObjectURL(xblob);
a = document.createElement("a");
document.body.appendChild(a);
a.style = "display: none";
a.href = url;
a.download = "audio_idea.ogg";
a.click();
setTimeout(() => {
document.body.removeChild(a);
window.URL.revokeObjectURL(url);
}, 100);
//window.URL.revokeObjectURL(url);
//console.log("downloading 1");
//amediaRecorder.start(1000);
//console.log("recording4 vs " + amediaRecorder.state);
}
}

function dostopit() {
astopCapture();
}

//setTimeout(dostopit, 9000);
</script>
</body>
</html>

This proved a good way to then (have that code, largely) ease into the changed screen_capture_api_va.htm Screen Capture API using video and audio downloadable content web application for you to try below, maybe?

In order to assist with user understanding of functionality we do a bit of …


<button onclick="download();" id=download disabled>Download &#128249;</button>

… HTML (design based) button disabling, teamed with Javascript, dynamic …


document.getElementById('download').disabled=false;

… button re-enabling, as required.

In the HTML “static design” side of “media play back” things we add to the bottom of the document.body element …


<hr id=myhr>
<table id=myplayt style="width:96%;"><tr><td style="width:70%;">
<video id=playvideo style='display:none;width:95%;' type='video/webm' onloadeddata='playa();' onclick='playa();' controls>
<!--source id=playvideov type='video/webm' style='display:none;' src=''></source-->
<!--source id=playvideoas type='audio/ogg' style='display:none;' src=''></source-->
</video></td><td style="width:30%;">
<audio id=playvideoa style='display:none;width:95%;' type='audio/ogg' controls></audio><br><div id=divsync></div></td></tr></table><br><br><p id=lastp></p>

… is used by reworked “download” logic as per …


function download() {
//console.log("downloading 0");
// let dtracks = videoElem.srcObject.getTracks();
//tracks.forEach(track => mediaRecorder.addTrack(track, stream));
// dtracks.forEach(track => window.stream.addTrack(track));
//console.log("recording1 vs " + mediaRecorder.state);
//mediaRecorder.stop();
//console.log("recording2 vs " + mediaRecorder.state);
if (mediaRecorder) {
if (('' + mediaRecorder.state) != 'inactive') {
mediaRecorder.requestData();
}
//console.log("recording3 vs " + mediaRecorder.state);
blob = new Blob(recordedChunks, {
type: "video/webm"
});
url = URL.createObjectURL(blob);


if (1 == 11) {
RequestAsBlob(url,
function(vblob)
{
var vurl = URL.createObjectURL(vblob);

document.getElementById('playvideo').src = vurl;
document.getElementById('playvideo').play();
});
} else {
//document.getElementById('playvideov').src=url;
document.getElementById('playvideo').src = url;
document.getElementById('playvideo').play();
}
if (1 == 1) {
//document.getElementById('playvideov').style.display='block';
document.getElementById('playvideo').style.display='block';
document.getElementById('myplayt').style.backgroundColor='yellow';
document.getElementById('myplayt').scrollIntoView(); //location.href='#lastp'; //document.getElementById('lastp').scrollIntoView(); //document.getElementById('playvideo').scrollIntoView(); //location.href='#myhr';
window.scrollBy(0,120);
}
a = document.createElement("a");
document.body.appendChild(a);
a.style = "display: none";
a.href = url;
a.download = "screen_capture_api_va.webm";
a.click();
setTimeout(() => {
document.body.removeChild(a);
window.URL.revokeObjectURL(url);
window.scrollTo(0,document.body.scrollHeight);
}, 100);
//window.URL.revokeObjectURL(url);
//console.log("downloading 1");
//mediaRecorder.start(1000);
//console.log("recording4 vs " + mediaRecorder.state);
}
if (document.getElementById('adownload')) { adownload(); }
}

function adownload() {
//console.log("downloading 0");
// let dtracks = videoElem.srcObject.getTracks();
//tracks.forEach(track => mediaRecorder.addTrack(track, stream));
// dtracks.forEach(track => window.stream.addTrack(track));
//console.log("recording1 vs " + mediaRecorder.state);
//mediaRecorder.stop();
//console.log("recording2 vs " + mediaRecorder.state);
if (amediaRecorder) {
if (('' + amediaRecorder.state) != 'inactive') {
amediaRecorder.requestData();
}
//console.log("recording3 vs " + amediaRecorder.state);
xblob = new Blob(chunks, {
type: "audio/ogg; codecs=opus"
});
aaurl = URL.createObjectURL(xblob);
if (1 == 11) {
RequestAsBlob(aaurl,
function(aablob)
{
var aaaurl = URL.createObjectURL(aablob);

document.getElementById('playvideoa').src = aaaurl;
document.getElementById('playvideoa').play();
});
} else {
//document.getElementById('playvideoa').src=url;
document.getElementById('playvideoa').src = aaurl;
if (delaya == 0) {
document.getElementById('playvideoa').play();
} else {
document.getElementById('divsync').innerHTML='<br><br>Clicking within video synchronizes an audiovisual replay';
document.getElementById('playvideo').title='Clicking within this video (not clicking play button) synchronizes an appropriate audiovisual replay';
playa(); //setTimeout(function(){ document.getElementById('playvideoa').play(); }, delaya);
//document.getElementById('playvideo').addEventListener('click', playa);
}
}
if (8 == 8) {
document.getElementById('playvideoa').style.display='block';
document.getElementById('playvideo').style.display='block';
}
aa = document.createElement("a");
document.body.appendChild(aa);
aa.style = "display: none";
aa.href = aaurl;
aa.download = "screen_capture_api_va.ogg";
aa.click();
setTimeout(() => {
document.body.removeChild(aa);
window.URL.revokeObjectURL(aaurl);
}, 100);
//window.URL.revokeObjectURL(url);
//console.log("downloading 1");
//amediaRecorder.start(1000);
//console.log("recording4 vs " + amediaRecorder.state);
}
}

function playa() {
setTimeout(function(){ document.getElementById('playvideoa').play(); }, delaya);
}

… the Javascript variable “delaya” containing the milliseconds between the “video start click” and the “audio start click” used in a setTimeout timer approach to synchronizing the “audio stream” play endpoint line up with the “video stream” play endpoint in these media elements above, even if a replay is achieved via a click “within” the video element (rather than using that video’s play button, where behaviour is as per usual).

Previous relevant Screen Capture API Download Video Tutorial is shown below.

Screen Capture API Download Video Tutorial

Screen Capture API Download Video Tutorial

Lots of users will have noticed regarding yesterday’s Screen Capture API Primer Tutorial creation of a …

  • Screen Capture API inspired “streamed video” … the downside is that …
  • context (or right-click or two finger gesture) menu over that “streamed video” has a grayed out “Save Video As…” option … hampering sharing, except that, today, we add …
  • “streamed video” sharing mechanism, featuring …
    1. one new HTML “Download 📹” button and canvas …

      <button onclick="download();" id=download>Download &#128249;</button>
      <canvas id=canvas></canvas>

      … linking to “onclick” logic …
    2. Javascript event logic for download of a “webm” video …

      var mediaRecorder=null;
      var blob=null, url=null, a=null;
      var recorderChunks=[];

      function download() {
      if (mediaRecorder) {
      if (('' + mediaRecorder.state) != 'inactive') {
      mediaRecorder.requestData();
      }
      blob = new Blob(recordedChunks, {
      type: "video/webm"
      });
      url = URL.createObjectURL(blob);
      a = document.createElement("a");
      document.body.appendChild(a);
      a.style = "display: none";
      a.href = url;
      a.download = "screen_capture_api_test.webm";
      a.click();
      setTimeout(() => {
      document.body.removeChild(a);
      window.URL.revokeObjectURL(url);
      }, 100);
      }
      }

      … using …
    3. Media Recorder object and method logics …

      async function startCapture() {
      recordedChunks = [];


      if (typeof(logElem) !== 'undefined') {
      logElem.innerHTML = "";
      }

      try {
      if (typeof(videoElem) !== 'undefined') {
      videoElem.srcObject = await navigator.mediaDevices.getDisplayMedia(displayMediaOptions);
      dumpOptionsInfo();
      }
      } catch(err) {
      console.error("Error: " + err);
      }

      if (typeof(stream) === 'undefined') {
      const canvas = document.querySelector("canvas");
      const stream = canvas.captureStream(25);

      window.stream = stream;
      const options = { mimeType: "video/webm;codecs=vp9" }; // vs 9
      mediaRecorder = new MediaRecorder(videoElem.srcObject, options);

      mediaRecorder.ondataavailable = function(e) {
      if (e.data.size > 0){
      recordedChunks.push(e.data);
      }
      };

      mediaRecorder.start(1000);


      mediaRecorder.onstop = function(e) {
      if (recordedChunks) {
      if(1 == 5 && recordedChunks.length > 0 && document.getElementById('audio')) {
      //const audio = document.querySelector('audio');
      audio.controls = true;
      bloba = new Blob(recordedChunks, { 'type' : 'audio/ogg; codecs=opus' });
      audioURL = window.URL.createObjectURL(bloba);
      audio.src = audioURL;
      }
      }
      location.href=document.URL;
      };

      }


      if (document.getElementById('mybody')) {
      document.getElementById('mybody').style.backgroundImage='none';
      }
      }

… in a changed screen_capture_api_test.htm‘s live run, that has a sharing component now, which you can also try below.

Previous relevant Screen Capture API Primer Tutorial is shown below.

Screen Capture API Primer Tutorial

Screen Capture API Primer Tutorial

We’ve spent another day researching the web Screen Capture API whereby …

The Screen Capture API introduces additions to the existing Media Capture and Streams API to let the user select a screen or portion of a screen (such as a window) to capture as a media stream. This stream can then be recorded or shared with others over the network.

Teamed with this great resource was the tutorial, we followed to a letter, called Using the Screen Capture API that had us being able to display a video of the Screen Capturing, including cursor movement, as applicable. Cute, huh?!

Again, as with many web API functionality, permissions may need to be tweaked allowing the web browsers involved, permission to access “Screen Capturing”, at least in the case of the macOS environment we tested this on today.

Luckily, we can show you the fruits of following a tutorial assiduously with screen_capture_api_test.html‘s live run you can also try below …

If this was interesting you may be interested in this too.

If this was interesting you may be interested in this too.

If this was interesting you may be interested in this too.

If this was interesting you may be interested in this too.

This entry was posted in Animation, eLearning, Event-Driven Programming, Tutorials and tagged , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *