Research & Projects
This page consists of my highlight music technology projects I worked on over the last few years. Majority of the projects involve Hyper-Instruments design, audio signal processing based projects and digital & analog music visualizers (few signal processing based, and some completely hardware). My current interests are in the lines of AI based music production tools, Automatic Mixing and Spatial Audio, particularly Binaural Sound Synthesis, Directivity Modelling, Audio Up-mixing and Automatic Spatialization.
Reports and implementations to all my projects and research are available on request!
Stereo to 5.1 Surround Audio up-mixing using time-Frequency Domain techniques
Designed an STFT analysis-synthesis model to perform up-mixing by deriving relevant statistical parameters from the stereo signal to create an immersive feel and produce a stable center channel for off-axis listening. The core techniques were ambience extraction and source re-panning by computing instantaneous inter-channel coherence and similarity measures.
An automatic mixing system for multitrack spatialization for stereo based on unmasking & panning practices
Designed and developed a novel automatic mixing algorithm to spatialize sound sources of a multitrack recording for masking minimization. Utilized time-frequency analysis techniques: spectral decomposition, sub-grouping, frequency based spreading, particle swarm optimization based on MPEG Psychoacoustic Masking Model.
Structural modelling for Binaural Sound Synthesis, Geometrical Acoustics & Directivity in Ambisonics
(Ongoing): Conducting research on building a parametric HRTF model based on time-domain description of the physics of wave propagation and diffraction for binaural audio playback, HRTF individualization and auralization using diffuse rendering. Investigating efficient methods to render directivity patterns for sound sources in Ambisonics.
Audio Denoising using Sparse Approximation Techniques in Time-Frequency domain
Implemented, compared and evaluated two sparse approximation techniques based on greedy as well as relaxation methods: Lasso-based regression and Orthogonal Matching Pursuit, for audio denoising.
Sensor fusion for IMUs : Real-time Quaternion-Based Orientation Estimation using Kalman Filter
Explored the orientation estimation problem for IMUs using sensor fusion with extended Kalman filter, Mahony’s nonlinear complementary filter and AHRS algorithm. Achieved a robust low-latency system implementation with minimal drift.
VoxNets - AI based Audio Querying and Sound Retrieval by Vocal Imitation Recognition
Ranked 3rd at ImplementAI Hackathon: Our team designed an audio querying system using Adverserial Autoencoder and Convolutional Neural Network to retrieve instrument class and pitch based on vocal imitation. Human voice input was segmented and analyzed using onset detection algorithms and constant-Q transform. (Librosa, Scipy, numpy, Torch, Keras)
'BreXting': Deep-learning based Brain-Computer Interface (BCI) on Intel DE-10 Nano FPGA
Intel InnovateFPGA Design Competition: Rank 2, World Grand Finals - Top 5
Aim: To use Electroencephalography (EEG) for decoding motor tasks thought by the user, to move an on-screen pointer thus enabling typing or drawing, or ‘BreXting’, using a standalone system with just an FPGA and a custom portable EEG headset.
Current Implementation: Trained a Convolutional Neural Network (CNN) with the BNCI Horizon 4-Class Motor Imagery dataset, the CNN architecture along with the stored weights are implemented on an FPGA, whose outputs trigger mouse pointer movements on a PC via JTAG.
Haptic Tutor: Wireless Vibro-tactile system to lower threshold for learning musical instruments
Ranked 1st at World Music Hackathon, MTF - Sweden
A novel music education tool that consists of a network of self-contained ERM vibration motors hooked onto Atmega328p micro-controllers. nRF24L01 RF modules enabled bi-directional communication of MIDI messages using which the teacher triggers timed vibrations on the haptic actuators strapped onto the limbs of the student drummer.
Unlisted video: available on request!
SMART Midi Band
SMART Band involves multiple wearable MIDI devices/hyper-instruments connected in a local network using various M2M protocols and MIDI protocol connected to a central DAW(Digital Audio Workstation).
Sound Engineering Experience
Behind the scenes of some of my projects...
I underwent multiple internships in the these domains through which I developed interest in IoT (Internet of Things) as it involved my favorite domains including Networking, Embedded Systems and Internet Technologies concepts. Being a Cisco Certified Network Associate (CCNA), I got several opportunities to lend my skills as a Network Engineer and gain relevant industrial experience. Last summer I got to be part of one of the most renowned network research groups at TU Berlin with the Telecommunications Networks group. I carried out research on improving the standard compliance of the 6TiSCH protocol stack [TSCH parts of IEEE 802.15.4e Standard] by adding support for automatic network configuration.
Here's a briefing about some of my projects in the lines of communication networks. My research interests revolve around the following:
PHY-MAC Layer Cross Layer Optimization
Computer Networks Infrastructure Design
Wireless Sensor Networks
Bluetooth Low Energy (BLE)
TSCH, PTP, Tiny OS
IoT (Internet of Things)
Indoor Positioning System (IPS) using Bluetooth Low Energy (BLE) :
Designed an Indoor Positioning System using BLE technology (HM-10 Modules with CC2541 BLE Chipset as beacons). A Flask based backend server was set up to receive JSON objects from user's smartphone running an android app which sent RSSI (Signal strength) values for the trilateration algorithm. Front end GUI displayed location based on the XY co-ordinates received from flask server after processing. The BLE nodes network implementation and architecture was designed in such a way that the project can be scaled for IoT applications.
Design and Evaluation of a Distributed out-of-Band Tracing Framework for Time-Sensitive systems:
Provided support for the the development of necessary hardware and software infrastructure for precise and out-of-band assessment of the end-to-end delay and jitter in the TWIST testbed. One of the major goals is to discipline the linux-based beagle bone black (PTP Server) using an external GPS-disciplined precise time source.
Extension of the 6TiSCH protocol stack for Internet of Things(IoT) Standards:
My research work involves further extension of the 6TiSCH protocol stack(TSCH parts of the IEEE 802.15.4e standard) in TinyOS developed as part of the EIT Digital activity RICH on Reliable IP-based wireless protocol stacks for demanding Cyber-Physical Systems. The work is aimed at improving the standard compliance of the stack by adding support for automatic network configuration based on parsing info from Information Elements in the packet headers.
I am also working on experimental verification of interoperability with the implementations in Contiki and OpenWSN by carrying out appropriate benchtop tests as well as larger experiments performed in TKN (TU Berlin)'s TWIST testbed.
Paper Publication - IEEE xplore (ICAEESE'16 Malaysia):
International Conference on Advances in Electrical, Electronic & Systems Engineering
Data Transfer Using MCM Code
Developed a digital watermarking based matrix type symbol to enable point-to-point communication between hand-held devices.
Multicolored Matrix (MCM) Code is a two dimensional matrix type symbol, consisting of colored squares, that is arranged as an image sequence rendered at an optimum frame rate to transfer simple data between devices using sender’s screen display and receiver’s camera. MCM code introduces an untried channel-less short range communication technology, designed with limitless storage capacity, great structural stability and flexibility due to which it can work in harsh lighting environments and still combat readability issues in the decoding process. The idea, inspired by the existing QR code, has various code types in terms of number of colors and size of matrix depending on length of the data, in the code generation and encoding process. This paper introduces our new idea of data transfer, the technical features of the code along with the encoding-decoding techniques, bit error rates and future scope of improvements for the idea.
Other Projects & Research:
Network Address Translation
Integrated static NAT/PAT to provide access to services located on a server in the private network to the public network. Implemented standard and extended access-lists to filter network traffic.
Virtual Private Network
Configured a site-to-site IPSec VPN using AES encryption algorithm, SHA-1 hash algorithm and RSA authentication method. Verified operation of IPsec at the network layer, protection and authentication of IP packets between participating IPsec devices and transmission of sensitive information over unprotected networks.
Configured Cisco routers for Syslog, NTP, SSH operations and enabled AAA authentication with the help of TACACS servers. Verified firewall functionality from internal and external hosts after implementing Zone-Based Policy Firewall.
Energy Analysis of Mesh Networks
Computer Communication Networks Minor Project: Conducted Energy Analysis of mesh networks and got insight about the duration of effective operation of the nodes. NS3 (open source network simulator) and Wireshark (packet sniffer) were used to carry out our experiments.
Rate Adaptation Algorithm and Cross Layer Design Optimization for TCP Wireless
Bachelor Thesis Project (Ongoing)
Goals of the research involves improving upon existing RAAs and realize an effective cross layer approach involving the TCP stack in order to achieve enhanced performance over wireless channels, particularly in ”dense” scenarios.
Here's a briefing about some of my projects in the lines of Embedded Systems. Most of my research and projects in this domain revolve around designing micro-controllers based wearable devices, developing various M2M communication protocols and Robotics. I have gained quality industrial and research experience through several workshops and internships. I am currently the Technical Head of the "Charge" SIG of ISTE (Indian Society for Technical Education), one of the premier tech clubs in the region. I organized prolific workshops and talks for students in the field of micro-controllers(RaspberryPi, Arduino, MSP 430, BeagleBoneBlack, etc.), computer architecture, communication networks.etc. and head several embedded systems projects.
Click the above links to view some of my embedded systems projects in Music Technology and Communication Networks.
World Robotics Olympiad
I represented my country to participate for the World Robotics Olympiad (WRO) for 3 consecutive years at WRO - Doha, WRO - AbuDhabi and WRO - Malaysia and secured Rank 1 (Gold) and World Rank 25 (Best Round Rank 14) respectively.
I have also conducted several robotics workshops and competitions in my college and schools nearby. I mentor several Robotics groups in the region.
IEEE Project Expo
Guide Stick for the Visually Impaired
Designed a guide stick that can detect obstacles using HCSR04 Ultrasonic sensors placed at head, torso and foot levels interfaced with MSP430G2553 Micro-controller, which gives feedback through DC vibration motors attached to the stick.
An accelerometer was mounted on top of the stick to detect any drastic shake/vibration to detect the stick falling. This would trigger an emergency interrupt to the user's smartphone via an HC 05 Bluetooth Module. The app on the smartphone receives these control signals to generate an emergency alert through SMS by sending the GPS co-ordinates of the user to the pre-chosen contact.
LED Matrix Screen
Designed and soldered a POV based 8 x 32 LED Matrix screen that can display text and simple bitmap images. The circuit involves a TLC5916 array that drives the LEDs and this is controlled my TI's MSP430 G2553 micro-controller. The design was further modified to sync to music and later built a 3D version of the same matrix. All previous functionalities were adopted to this design in which games like 3-D snake and simple EQ based music sync could be displayed.
Voice controlled Room running on IoT framework
I connected all electrical appliances (light bulbs, fans, switches, etc) in my room to opto-coupler traic pairs and 8 Channel Relays, whose inputs were fed by a state-machine based script run on Raspberry Pi(RPi). The RPi hosts the server that has a speech recognition module which requires voice commands as input. A python script is run to do text mining on the recognised text from user's speech. The commands can control back-light fit on the table, bed and other furniture. The LED strips array is configured to sync to music played on the server.
Moving object tracking - SWARM Robots & 6th Sense Technology
Developed an application based on 6th sense technology in which the following operations could be done:
-Control the mouse pointer by hovering user's finger in front of webcam
-Scrub through media player and adjust volume control using hand gestures
-Control a robot using hand gestures
Used space domain image processing tools in matlab and communication to bot was established using serial communication(winAVR programming) further building the following project:
Designed and constructed a swarm of robots(mounted with webcams) programmed to track a moving ball. Wireless communication between the bots was established using RF modules(CC2500) which worked on SPI protocol.