Heavy load cracks on Firefox and CPU never comes back flat #64

dengarcia · 2018-09-30T22:20:41Z

Hi have spent some time trying to understand where the difference comes from in Firefox and Chrome and I am now reaching to you for help.

I run the following code that works well on Chrome (no crack and CPU comes back near flat after the test) but in Firefox it cracks and CPU is high and never goes back down even a few minutes after the test is finished.

Is that a bug in Firefox or is that a bug in the library?

The code will execute play on the instrument (woodblock) every 0.1 second for 20 seconds.

<html>
	<head>
		<title>test heavy load</title>
		<script type="text/javascript" src="soundfont-player.min.js"></script>
		<script type="text/javascript">
			var audioContext = new AudioContext();
            Soundfont.instrument(audioContext, 'woodblock', {
                    destination: audioContext.destination,
                    loop: false,
                    notes: ["C4"] })
                .then(function (instrument) {
                    // console.log("instrument loaded");
                    var interval = 20; // in second
                    var iter = 10 * interval;
                    for(var i=0 ; i<iter ; i++) {
                        var time = audioContext.currentTime + interval/iter * i;
                        instrument.play("C4", time, time + 1);
                    }
                });
		</script>
		</head>
	<body>
	TEST PAGE
	</body>
</html>

The text was updated successfully, but these errors were encountered:

dengarcia · 2018-11-02T08:02:26Z

Hello, did anyone have a look to this issue. The soundfont-player is not usable with Firefox, and I have not been able to identify why. I will have to stop using the player and use one I built. Sad because this one is a bit more flexible than mine.

danigb · 2018-11-02T08:30:06Z

I'll take a look at this issue today. Can you specify your OS version and Firefox version? Have you tested in other browsers? Any other information that could be relevant will be welcomed.

dengarcia · 2018-11-02T10:37:19Z

Hello, indeed I should have given more information about my tests

I have tested with:

Chrome 70+ on Windows 10
Firefox 63.0.1

I have not seen the issue on Chrome 10 (neither cracks not CPU issue)
I have also been told (but could not test myself) that the issue does not happen on Opera with IOS 12.1

Let me know if you need more information or if you need me to do more investigations

Thanks

danigb · 2018-11-03T21:43:11Z

Hi @dengarcia

I've tested the exact code you posted before on Firefox (v63.0.1) and worked exactly the same as in Chrome. I don't know what's your problem, but I couldn't reproduce it.

Anyway, I have to say that scheduling capabilities of this library are very limited (to say it as best). Instead of scheduling 200 notes in advance, it's better to have a function that is called every second (for example) and just schedule the 10 notes of that second. You can find more info here: https://www.html5rocks.com/en/tutorials/audio/scheduling/

dengarcia · 2018-11-04T18:07:01Z

Hello,

Thank you for your reply.

There is a possibility it depends on the computer and the environment, that said I have tested on three computers with high and low spec, and consistantly found the same issue.

Here is an improved example showing the issue consitantly for me on Firefox and not on Chrome

click on "Create AudioContext"
click on "Start Test1" (this is the one using soundfont-player)
let it run. The test stops after 10 minutes, but on my computer (good developer spec) it takes 50 secondes to start hering cracks, by then the CPU is at around 30% with Firefox, but 2% on Chrome.
When I click stop. The CPU never goes down with Firefox but since it never went up on Chrome it is low on Chrome.
On both browser the CPU is low after I close the AudioContext (click "Close AudioContext")

I have added Test2 which loads the same sound and plays it in a buffer I create with AudioContext. There is no issue at all with Test2

So I believe there is something done in the soundfont-player that adds something that breaks in Firefox.

I will try to investigate in detail again in soundfont-player. But let me know if you still don't see the issue with this test.

<!doctype html>
<html>
	<head>
		<title>test heavy load</title>
		<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
		<script type="text/javascript" src="soundfont-player.min.js"></script>
		<script type="text/javascript" src="https://gleitz.github.io/midi-js-soundfonts/MusyngKite/woodblock-mp3.js"></script>
		<style>
			div { padding: 6px; }
			#message { color: blue; }
			#code { text-align: left; color: red; border: 1px solid red; }
		</style>
		</head>
	<body>
		<h1>TEST PAGE</h1>

		<div>
		<button id="recreateAudioContext">Create AudioContext</input>	
		<button id="closeAudioContext">Close AudioContext</input>	
		</div>
		
		<div>
		<button id="startTest1">Start Test1</input>	
		<button id="startTest2">Start Test2</input>	
		<button id="stopTest">Stop Test</input>	
		</div>

		<div id="message"></div>

		<script type="text/javascript">
			function test1(audioContext, seconds) {
				Soundfont.instrument(audioContext, 'woodblock', {
						destination: audioContext.destination,
						loop: false,
						notes: ["C4"] })
					.then(function (instrument) {
						interval = setInterval(function() {
							var iter = 10; // Number of hits per second
							for(var i=0 ; i<iter ; i++) {
								var time = audioContext.currentTime + i/iter + 0.5;
								instrument.play("C4", time, time + 1);
							}
							seconds--;
							if(seconds <= 0) {
								stopTest();
							}
						}, 1000);
					});
			}

			function test2(audioContext, seconds) {
				var gainNode = audioContext.createGain();
				gainNode.gain.setTargetAtTime(0.5, audioContext.currentTime, 0.2);
				gainNode.connect(audioContext.destination);
				
				interval = setInterval(function() {
					var iter = 10; // Number of hits per second
					for(var i=0 ; i<iter ; i++) {
						var time = audioContext.currentTime + i/iter + 0.5;
						var bufferNode = this.audioContext.createBufferSource();
						bufferNode.buffer = woodblock;
						bufferNode.loop = false;
						bufferNode.connect(gainNode);
						bufferNode.start(time);
						bufferNode.stop(time + 0.5);
					}
					seconds--;
					if(seconds <= 0) {
						stopTest();
					}
				}, 1000);
			}
			</script>

		<script type="text/javascript">
			var audioContext = null;
			var interval = null;

			function startTest1() {
				if(!audioContext) {
					document.getElementById("message").innerHTML = "No AudioContext";
					return;
				}
				test1(audioContext, testTime);

				startTest1Button.disabled = true;
				startTest2Button.disabled = true;
				stopTestButton.disabled = false;
			}
			
			function startTest2() {
				if(!audioContext) {
					document.getElementById("message").innerHTML = "No AudioContext";
					return;
				}
				test2(audioContext, testTime);

				startTest1Button.disabled = true;
				startTest2Button.disabled = true;
				stopTestButton.disabled = false;
			}
			
			function stopTest() {
				console.log("stopTest");
				if(interval) {
					clearInterval(interval);
					interval = null;
				}

				startTest1Button.disabled = false;
				startTest2Button.disabled = false;
				stopTestButton.disabled = true;
			}
			
			function recreateAudioContext(e) {
				closeAudioContext();
				audioContext = new AudioContext();

				startTest1Button.disabled = false;
				startTest2Button.disabled = false;
				stopTestButton.disabled = true;
				recreateButton.disabled = true;
				closeButton.disabled = false;
			}

			function closeAudioContext(e) {
				if(audioContext != null) {
					audioContext.close();
					audioContext = null;
				}

				startTest1Button.disabled = true;
				startTest2Button.disabled = true;
				stopTestButton.disabled = true;
				recreateButton.disabled = false;
				closeButton.disabled = true;
			}
			
			function loadWoodblock() {
				//var url = "https://gleitz.github.io/midi-js-soundfonts/MusyngKite/woodblock-mp3.js";
				var c4 = MIDI.Soundfont.woodblock.C4;
				var i = c4.indexOf(',')
				c4 = decode(c4.slice(i + 1));
				c4 = decode(c4);
				var tmpAudioContext = new AudioContext();
				tmpAudioContext.decodeAudioData(c4.buffer,
					function(buffer) {
						woodblock = buffer;
						tmpAudioContext.close();
					},
					function(error) {
						console.error("error: " + error);
					}
				);
			}
			
			function b64ToUint6 (nChr) {
				return nChr > 64 && nChr < 91 ? nChr - 65
					: nChr > 96 && nChr < 123 ? nChr - 71
					: nChr > 47 && nChr < 58 ? nChr + 4
					: nChr === 43 ? 62
					: nChr === 47 ? 63
					: 0;
			}

			function decode(sBase64, nBlocksSize) {
				var sB64Enc = sBase64.replace(/[^A-Za-z0-9\+\/]/g, '')
				var nInLen = sB64Enc.length
				var nOutLen = nBlocksSize
					? Math.ceil((nInLen * 3 + 1 >> 2) / nBlocksSize) * nBlocksSize
					: nInLen * 3 + 1 >> 2
				var taBytes = new Uint8Array(nOutLen)

				for (var nMod3, nMod4, nUint24 = 0, nOutIdx = 0, nInIdx = 0; nInIdx < nInLen; nInIdx++) {
					nMod4 = nInIdx & 3
					nUint24 |= b64ToUint6(sB64Enc.charCodeAt(nInIdx)) << 18 - 6 * nMod4
					if (nMod4 === 3 || nInLen - nInIdx === 1) {
					for (nMod3 = 0; nMod3 < 3 && nOutIdx < nOutLen; nMod3++, nOutIdx++) {
						taBytes[nOutIdx] = nUint24 >>> (16 >>> nMod3 & 24) & 255
					}
					nUint24 = 0
				}
			  }
			  return taBytes
			}

			
			var testTime = 60; // seconds
			var woodblock = null;
			loadWoodblock();
			
			var startTest1Button = document.getElementById("startTest1");
			var startTest2Button = document.getElementById("startTest2");
			var stopTestButton = document.getElementById("stopTest");
			var recreateButton = document.getElementById("recreateAudioContext");
			var closeButton = document.getElementById("closeAudioContext");
			
			startTest1Button.addEventListener("click", startTest1);
			startTest2Button.addEventListener("click", startTest2);
			stopTestButton.addEventListener("click", stopTest);		
			recreateButton.addEventListener("click", recreateAudioContext);
			closeButton.addEventListener("click", closeAudioContext);
			
			startTest1Button.disabled = true;
			startTest2Button.disabled = true;
			stopTestButton.disabled = true;
			recreateButton.disabled = false;
			closeButton.disabled = true;

		</script>

	</body>
</html>

Denis

finngl · 2019-06-10T20:17:57Z

I experience this problem, too. In Firefox, when the audio becomes just moderately stressed, the sound becomes crackled. I don't schedule sounds, just play them in real time. My environment is Windows 10 Personal Edition.

kenakofer · 2021-01-07T15:52:01Z

Edit: This comment was on an older version!

I experience this problem, too.

On Firefox 84.0.2 (64-bit) on Mac Catalina 10.15.7: In activity monitor, I see the process FirefoxCP file:// Content cpu usage creep upwards as playback continues, and not fall back down when playback is stopped. Crackling of all computer audio is gradually introduced after about 1 minute of playing, as CPU usage increases. Refreshing the webpage causes cpu to drop back down and crackling to be fixed.

On Chrome 87.0.4280.88 (Official Build) (x86_64), CPU stays low given the same actions and timescale, and there is no crackling

kenakofer · 2021-01-09T17:57:20Z

I was not on the latest version of soundfont-player, and can't replicate the bug in the newest version. My apologies for posting prematurely on something that is likely already fixed.

hylu-dev · 2021-02-07T04:05:39Z

Hmm I'm not sure, I've only recently started using this soundfont player last week to make a simple piano and I'm running into the exact issue as kenanbit described when I use Firefox 85.0.1 (64-bit) on Windows 10. Chrome works fine.

inductiveload · 2021-11-11T11:17:27Z

I have exactly the same issue on Arch, Firefox 94.0.

I used this for a toy project: https://ws-songbird.toolforge.org/ and if you turn on the Wikidata events, the sounds eventually goes crackly and stays that way.

In the example at the top of this issue, setting var interval = 20; // in second to something like 100 makes the situation much worse than increasing the rate and reducing the interval. So it feel like it's an issue with the total number of queued events rather than the number of simultaneous events.

mouse-reeve mentioned this issue Nov 2, 2018

Firefox memory leak mouse-reeve/infinite_gnossienne#5

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Heavy load cracks on Firefox and CPU never comes back flat #64

Heavy load cracks on Firefox and CPU never comes back flat #64

dengarcia commented Sep 30, 2018 •

edited

dengarcia commented Nov 2, 2018

danigb commented Nov 2, 2018 •

edited

dengarcia commented Nov 2, 2018

danigb commented Nov 3, 2018

dengarcia commented Nov 4, 2018

finngl commented Jun 10, 2019

kenakofer commented Jan 7, 2021 •

edited

kenakofer commented Jan 9, 2021

hylu-dev commented Feb 7, 2021 •

edited

inductiveload commented Nov 11, 2021

Heavy load cracks on Firefox and CPU never comes back flat #64

Heavy load cracks on Firefox and CPU never comes back flat #64

Comments

dengarcia commented Sep 30, 2018 • edited

dengarcia commented Nov 2, 2018

danigb commented Nov 2, 2018 • edited

dengarcia commented Nov 2, 2018

danigb commented Nov 3, 2018

dengarcia commented Nov 4, 2018

finngl commented Jun 10, 2019

kenakofer commented Jan 7, 2021 • edited

kenakofer commented Jan 9, 2021

hylu-dev commented Feb 7, 2021 • edited

inductiveload commented Nov 11, 2021

dengarcia commented Sep 30, 2018 •

edited

danigb commented Nov 2, 2018 •

edited

kenakofer commented Jan 7, 2021 •

edited

hylu-dev commented Feb 7, 2021 •

edited