Fragment Synthesizer : GLSL powered HTML5 spectral synthesizer

Some years ago i found out the Virtual ANS synthesizer, this is a very good spectral synthesizer simulating the Russian ANS synthesizer, an unique photoelectronic musical instrument where the score is a drawn sound spectrogram, the x axis of the score represent time and the y axis represent frequency.

Since then i have a great interest in this sort of synthesis and method of composing and have an ongoing large project heavily related to the Virtual ANS.

Fragment Synthesizer is a fun side experiment made quickly where the initial thought was : What if you set GLSL produced images as the source of a spectral synthesizer?

The result is the Fragment Synthesizer web application

This is a full blown stereophonic (the color matter, red for left, green for right) spectral synthesizer which is constantly playing a slice of a GLSL produced image/animation, you can compose by editing the fragment shader or by just copy-pasting code from Shadertoy and then convert it to the Fragment Synthesizer format by clicking on the convert button.

The web app. consist of 3 parts :

  • The score produced by a fragment shader with a vertical bar representing the slice which will be played by the synthesizer
  • A live code editor with the ability to compile as you type (with errors reporting), this is used to edit the GLSL fragment shader and subsequently to compose. (the code editor is powered by the CodeMirror library)
  • Controls (volume slider, button to convert Shadertoy code to Fragment Synthesizer code and a slider to move the playing slice) powered by my own JavaScript widget library

How does it work?

Audio side :

The Fragment Synthesizer is just a simple additive synthesizer under the hood, it is powered by a simple wavetable which is generated with this code :

        _wavetable_size = 32768,
        
        _wavetable = (function (wsize) {
                var wavetable = new Float32Array(wsize),

                    wave_phase = 0,
                    wave_phase_step = 2 * Math.PI / wsize,

                    s = 0;

                for (s = 0; s < wsize; s += 1) {
                    wavetable[s] = Math.sin(wave_phase);

                    wave_phase += wave_phase_step;
                }

                return wavetable;
            })(_wavetable_size),

There is then an oscillator for each lines of the score, oscillators are generated by this function :

    var _generateOscillatorSet = function (n, base_frequency, octaves) {
        var y = 0,
            frequency = 0.0,
            octave_length = n / octaves;
        
        _oscillators = [];

        for (y = n; y >= 0; y -= 1) {
            frequency = base_frequency * Math.pow(2, y / octave_length);

            var osc = {
                freq: frequency,
                
                phase_index: Math.random() * _wavetable_size, 
                phase_step: frequency / _audio_context.sampleRate * _wavetable_size
            };
            
            _oscillators.push(osc);
        }
    };

On the Fragment Synthesizer the starting frequency is 16.34 hertz (bottom of the score) and the y axis span 10 octaves, this is hardcoded but could be fun to let the user change it, the number of oscillators change as the score height change and it depend of the window height, if the user resize the window, the number of oscillators will change.

One of the most important function is the _computeNoteBuffer function, it will transform the pixels array of a vertical slice into an usable and fast to process data structure consisting of a single float32 typed array, each entries (an entry is in fact 5 entries because data is packed linearly into the array) of this array describe which oscillator to play along with data related to how it should play, here is what the 5 entries are :

  • The index of the oscillator to play
  • The previous left side gain value for this oscillator
  • The previous right side gain value for this oscillator
  • The current left side gain value for this oscillator
  • The current right side gain value for this oscillator

The gain value for each side is determined by the red and green component of the pixel value.

The previous gain value is used because it is interpolated when played, this produce a better sound without crackles when the gain value vary greatly.

This function is actually called in the audio callback (it is very fast so ok), it is called for each frames, here is how the number of samples before the next note is computed :

        _fps = 60,
        _note_time = 1 / _fps,
        _note_time_samples = Math.round(_note_time * _sample_rate),

Here is the code of the _computeNoteBuffer function :

    var _computeNoteBuffer = function () {
        for (i = 0; i < _note_buffer.length; i += 1) {
            _note_buffer[i] = 0;
        }
        
        var note_buffer = _note_buffer,
            pvl = 0, pvr = 0, pr, pg, r, g,
            inv_full_brightness = 1 / 255.0,

            dlen = _data.length,
            y = _canvas_height - 1, i,
            volume_l, volume_r,
            index = 0;

        for (i = 0; i < dlen; i += 4) { pr = _prev_data[i]; pg = _prev_data[i + 1]; r = _data[i]; g = _data[i + 1]; if (r > 0 || g > 0) {
                volume_l = r * inv_full_brightness;
                volume_r = g * inv_full_brightness;
                
                pvl = pr * inv_full_brightness;
                pvr = pg * inv_full_brightness;

                note_buffer[index] = y;
                note_buffer[index + 1] = pvl;
                note_buffer[index + 2] = pvr;
                note_buffer[index + 3] = volume_l - pvl;
                note_buffer[index + 4] = volume_r - pvr;
            } else {
                if (pr > 0 || pg > 0) {
                    pvl = pr * inv_full_brightness;
                    pvr = pg * inv_full_brightness;

                    note_buffer[index] = y;
                    note_buffer[index + 1] = pvl;
                    note_buffer[index + 2] = pvr;
                    note_buffer[index + 3] = -pvl;
                    note_buffer[index + 4] = -pvr;
                }
            }

            y -= 1;

            index += 5;
        }
        
        _prev_data = _data;
        
        _swap_buffer = true;
    };

Now the core audio code where the magic happen (nothing really fancy here except the interpolation) :

    var _audioProcess = function (audio_processing_event) {
        var output_buffer = audio_processing_event.outputBuffer,
            
            output_data_l = output_buffer.getChannelData(0),
            output_data_r = output_buffer.getChannelData(1),
            
            output_l = 0, output_r = 0,
            
            wavetable = _wavetable,
            
            note_buffer = _note_buffer,
            note_buffer_len = note_buffer.length,
            
            wavetable_size_m1 = _wavetable_size - 1,
            
            osc,
            
            lerp_t_step = 1 / _note_time_samples,
            
            sample,
            
            s, j;
        
        for (sample = 0; sample < output_data_l.length; sample += 1) {
            output_l = 0.0;
            output_r = 0.0;

            for (j = 0; j < note_buffer_len; j += 5) { var osc_index = note_buffer[j], previous_volume_l = note_buffer[j + 1], previous_volume_r = note_buffer[j + 2], diff_volume_l = note_buffer[j + 3], diff_volume_r = note_buffer[j + 4]; osc = _oscillators[osc_index]; s = wavetable[osc.phase_index & wavetable_size_m1]; output_l += (previous_volume_l + diff_volume_l * _lerp_t) * s; output_r += (previous_volume_r + diff_volume_r * _lerp_t) * s; osc.phase_index += osc.phase_step; if (osc.phase_index >= _wavetable_size) {
                    osc.phase_index -= _wavetable_size;
                }
            }
            
            output_data_l[sample] = output_l;
            output_data_r[sample] = output_r;
            
            _lerp_t += lerp_t_step;
            
            _curr_sample += 1;

            if (_curr_sample >= _note_time_samples) {
                _lerp_t = 0;

                _curr_sample = 0;

                _computeNoteBuffer();
            }
        }
    };
Visual side :

On the visual side, it is powered by WebGL and GLSL, there is a basic screen aligned quad with a fragment shader applied to it and then for each frames a readPixels call is made to get the user chosen (_play_position) vertical slice pixels array which will get converted in the audio callback by the computeNoteBuffer function, here is the code called for each frames :

    var _frame = function (raf_time) { 
        _gl.useProgram(_program);
        _gl.uniform1f(_gl.getUniformLocation(_program, "globalTime"), (raf_time - _time) / 1000);
        _gl.uniform2f(_gl.getUniformLocation(_program, "iMouse"), _mx, _my);

        _gl.drawArrays(_gl.TRIANGLE_STRIP, 0, 4);

        if (_swap_buffer) {
            _gl.readPixels((_canvas_width - 1) * _play_position, 0, 1, _canvas_height, _gl.RGBA, _gl.UNSIGNED_BYTE, _data);

            _swap_buffer = false;
        }

        _raf = window.requestAnimationFrame(_frame);
    };

And voilà, the core of the synthesizer explained.

Note : Borrowed from Shadertoy, the resolution, iMouse and globalTime uniforms are defined and can be used in the fragment shader to do the cool stuff! 🙂

The full source code is available on github

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

Simple Python automation script to include content inside files

Bored of writing the same text chunk for your online store products description or ever wanted to have an “include” directive in JavaScript?

Here is a tiny automation script made in Python, i use it to include repetitive content inside HTML files, it read all *.html files (the extension can be changed on line 30) line by line in the directory the script is executed from and any {include=filename} tag found is replaced by the file “filename” content, it is fully recursive.

Note : The script do not check for circularity… so do not include a file including itself or similar stuff 🙂

Note : I am also using a variant written in the Anubis language for JavaScript which include a simple #include filename directive, that variant is also a tiny automatic build system which look for changes in a directory and build automatically the application package consisting of a single JS and CSS minified file if a file has changed, that way i just focus on writing code, the build system produce everything in the background when it detect a change and to test the app i just have to refresh the page in the browser, i may share it one day but the code is uglier than this. 🙂

# -*- coding: utf-8 -*-
#!/usr/bin/python

import os
import re
import sys
import glob

include_regex = re.compile("{include=(.*?)}")

def get_line_content(file):
    content = ""
    current_line = 0
    
    with open(file, 'r') as f:
        for line in f:
            inc = include_regex.search(line)
            if inc:
                inc_filename = inc.group(1)
                content += get_line_content(inc_filename)

                print("including " + inc_filename + ", include directive found in file " + file + " at line " + str(current_line))
            else:
                content += line
            current_line += 1
        f.close()
    return content

os.chdir(".")
for file in glob.glob("*.html"):
    head, tail = os.path.split(file)
    
    output_filename = "output/" + tail
    
    content = ""
    
    print("processing " + file)
    
    content = get_line_content(file)
    
    print("producing " + output_filename)
    
    f = open(output_filename, 'w')
    f.write(content)
    f.close()

1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)
Loading...

How to remove malicious code infecting every PHP files of a website

A few month ago i stumbled upon some malicious replicating code (PHP.Anuna and PHP/Agent.GC) infecting all of my php files by inserting the same malicious code at the start of every files… slowing down all the website and gathering informations, this was a mess but this can be solved easily.

If you encounter malicious code from these or variants which are using the same signature i will show you how you can clean all of your website files very quickly and monitor it to prevent any other threats (look up the next article) so you can react before it is too late and let it to never happen again.

Here is a little PHP script to run on your server which recursively remove all the malicious code for every .php files in a specified folder based on a virus signature, it is very simple but also very effective for this kind of infection.

If there is any infections found it will show which file was cleaned up and at which position in the file the malicious code was, there is a report at the end of the decontamination process which will show you how much files were treated.

You likely have to configure it before using it because the signature will be different for each infections :

  • Change $target_directory value by the directory you want to clean up
  • Change $virus_begin_signature_str value by the few starting characters of the malicious code
  • Change $virus_end_signature_str value by the end of the malicious code

Disclaimer : I take no responsibility for any loss or damage suffered as a result of using this script, always backup your stuff first.

Note : Due to the use of the file_get_contents function, the script may get very memory hungry for large files, some adaptations of the code with more clever functions may be needed for your case.

<?php
/////////////////
$target_directory = '/var/www/';

//$virus_begin_signature_str = '$pmtccnmmns = \'341]88M4P8]37]278]225]241]334]';
$virus_begin_signature_str = 'd($n)-1);} @error_rP6]';
$virus_end_signature_str = '$pkvpncwqkl(""); $pkvpncwqkl=(468-347); $pmtccnmmns=$pkvpncwqkl-1; ?>';
$virus_name = 'PHP Agent.GC'; // just for pretty print, work with PHP.Anuna and variants
/////////////////

$recursive_directory_iter = new RecursiveDirectoryIterator($target_directory);
$iterator = new RecursiveIteratorIterator($recursive_directory_iter);

$decontaminated_files_count = 0;

foreach ($iterator as $filename => $cur)
{
	$path_info = pathinfo($filename);
	
	if (!isset($path_info['extension'])) {
        continue;
    }
        
    if ($path_info['extension'] !== 'php') {
		continue;
	}
	
	echo "Checking: '".$filename."'".PHP_EOL;
	
    $contents = file_get_contents($filename);
	
	$virus_begin_pos = strpos($contents, $virus_begin_signature_str);

    if ($virus_begin_pos !== false && $filename !== __FILE__) {
        echo $virus_name." found in '".$filename."'".PHP_EOL;
		
		$virus_end_str = $virus_end_signature_str;
		$virus_end_pos = strpos($contents, $virus_end_str) + strlen($virus_end_str);
		
		$before_virus_content = substr($contents, 0, $virus_begin_pos);
		$after_virus_content = substr($contents, $virus_end_pos);
		
		echo $virus_name." content between start pos ".$virus_begin_pos." and end pos ".$virus_end_pos." deleted".PHP_EOL;
		
		$contents = $before_virus_content.$after_virus_content;

        file_put_contents($filename, $contents);
		
		$decontaminated_files_count++;
    }
}

echo $decontaminated_files_count." files were infected and decontaminated in the directory '".$target_directory."'".PHP_EOL;
1 Star2 Stars3 Stars4 Stars5 Stars (6 votes, average: 5.00 out of 5)
Loading...