Forem: 𝓢𝓸𝓻𝓼𝓴𝓸𝓸𝓽🌷

WebXR at Global XR Conference

𝓢𝓸𝓻𝓼𝓴𝓸𝓸𝓽🌷 — Mon, 29 Nov 2021 20:50:56 +0000

I posted this to the WebXR discord (https://discord.gg/Jt5tfaM), but I thought it might be helpful to anyone interested outside of the Discord as well (but you should join 😉).

This week is the Global XR Conference (Dec 1 - 3) and there are a couple of WebXR related talks and workshops.

But I thought I'd make a list of all sessions and workshops that have the tag WebXR... Turned out be more than I thought 🙂 , but here they are:

December 1

2pm UTC - XR On The Web Is Just a Fad - Jonathan Hale - https://youtu.be/VCXe0Dm1tJs
2:30pm UTC - (Workshop) Building a VR Space Shooter - Timmy Kokke - https://youtu.be/zMI9-FpL4NI
4pm UTC - WebAR is Re-Inventing QR Codes - Frans Tihveräinen - https://youtu.be/jMFi-6qFg4I
8pm UTC - The Journey to Augmenting Your World - Ven Natarajan & Supun Bhagya - https://youtu.be/wa-nbNCwyc8

December 2

1pm UTC - (Workshop) Build a Multi-User Cross-XR App - Jonathan Hale - https://youtu.be/Ca1IvVk6OQo
1pm UTC - Exploring the Universe Using WebXR - Karan Balkar - https://youtu.be/4hUPVNGHhkA
2pm UTC - How to Build a High Scalable Microfrontend TWA Webxr App as a Single Native App - R Surahutomo Aziz Pradana - https://youtu.be/bDfS2IwJo-g
3pm UTC - (Community) Extended Reality in Africa: Present & Future Prospects - Arome Ibrahim - https://youtu.be/Yz3z7_2y-Ck
7:30pm UTC - (Workshop) Mastering the Immersion Platform - Ven Natarajan & Supun Bhagya - https://youtu.be/rELlPnFHkHw
9pm UTC - Meta Media Futures - Evo Heyning - https://youtu.be/4n-foFTqzcw

December 3

1pm UTC - The Rise of Virtual Worlds - Sonya Seddarasan & Jal Shah - https://youtu.be/JF4GZ0-8ZQk
4pm UTC - XR in Marketing and Advertisement - Diana Olynick - https://youtu.be/wBcDwolHT2c
7pm UTC - My Experience Adapting a Digital Twin WebGL Project for HoloLens 2 - Bruno Capuano - https://youtu.be/c3hxE1uvLvA

They have a discord as well if you like to reach out to one of the speakers ().

A-Frame hand-tracking on the HoloLens

𝓢𝓸𝓻𝓼𝓴𝓸𝓸𝓽🌷 — Sat, 24 Oct 2020 21:23:10 +0000

On the HoloLens you only have the UWP version of Edge out of the box. This doesn't have the newer WebXR device API implemented. The WebXR device API can even handle hand tracking. But, Edge UWP doesn't, this only has the old WebVR API. I already noticed you could use the Gamepad API to get an Air tap event and handle that as a click, but I ~~wanted~~ needed to get the position of the hands.

It took me a while to get this figured out this one. It turned out to be a bit easier than I thought. I thought I had to track the controls myself through the gamepad API, but it turned out A-Frame has us covered.

There's a component that is the basis for a lot of other components that handle tracking on various devices, for example, the Oculus touch controls or the Windows motions controls. If you know what ID is used in a specific situation you can just handle the rest yourself.

I created a very small test application and used Vorlon as I described in a previous post to get the information. I was implementing the whole thing myself at first, but later realized I could use the Tracked-Controls component that's in A-Frame. Below is a small snippet of HTML from that test app that shows how to use the left and right hand as a controller. The controller numbers 4 and 5 are actually the indices of the gamepads returned by the GamePad API. For the idPrefix I used Hand, which is short for 'Hand (Spatial Interaction Source)', the full id. I used the default low poly hand models used by A-Frame, but had to tweak the rotation and position a little bit to make them overlay the real hands a bit better.

<a-entity tracked-controls="controller: 4;hand:left; idPrefix: Hand">
    <a-entity id="hand" 
            rotation="-60 0 60" 
            position ="0 -.03 -.02 " 
            gltf-model="src:url(https://cdn.aframe.io/controllers/hands/leftHandLow.glb)">
        </a-entity>
</a-entity>
<a-entity tracked-controls="controller: 5;hand:right; idPrefix: Hand">
    <a-entity id="hand" 
            rotation="-60 0 -60" 
            position ="0 -.03 -.02 " 
            gltf-model="src:url(https://cdn.aframe.io/controllers/hands/rightHandLow.glb)">
        </a-entity>
</a-entity>

The only thing that this does not do is track the individual fingers. That's something you need the newer WebXR device API for, but this is better than nothing. I hope, one day, Edge on the HoloLens will support the full WebXR device API.

Next, I need to figure out a way of creating interaction between hands and objects in 3D space to be able to press buttons or touch objects. More on that in a future post.

WebXR Discord

𝓢𝓸𝓻𝓼𝓴𝓸𝓸𝓽🌷 — Sun, 27 Sep 2020 03:10:45 +0000

During the Covid-19 crisis all over the world, we noticed a lot of people from all over the world joining our meetups and members of our community hanging out in other meetups. We decided to try and bring everyone together in one Discord server. Here we can share links to our events and recordings, but also help each other out with issues when developing with A-frame, Babylon.js, or Unity and running your apps on devices like the Oculus Quest or Microsoft Hololens. Now we need you to join and help grow this community to become the official WebXR community.

Make sure to share the invite with your friends and colleagues: WebXR Discord, http://discord.gg/Jt5tfaM!

Remote debugging WebXR on HoloLens

𝓢𝓸𝓻𝓼𝓴𝓸𝓸𝓽🌷 — Sat, 26 Sep 2020 19:35:48 +0000

I recently started a new job as a Mixed Reality developer building applications for the Microsoft HoloLens. We build our applications using Unity3D. But, of course, I had to try and run WebXR on the device as well. Microsoft Edge comes with the HoloLens and supports WebVR. And switching to 'VR' removes the browser chrome and your 3D model becomes a hologram in the real world. After that, it gets different from running a WebXR app on other devices very quickly. One of the difficulties I ran into was debugging.

Debugging

When running a web application on the Oculus Quest, for example, you can use Edge on your desktop de remotely debug. Just browse to edge://inspect and without too much hassle you should be able to debug with the same tools you would debug any other website. The HoloLens runs the 'old' UWP version of Edge and can't be debugged with the new Chromium Edge. I've tried a couple of different approaches, but none worked.

Vorlon.JS

This is where Vorlon.JS comes in. This nifty little tool can replace the web debug tools and creates a connection to debug your website remotely. Just install the npm package globally on your machine (npm i vorlon -g). When you run this it spins up a small web server locally, by default on port 1337. You'll have to add a script that is served from this page to your HTML page and you are pretty much good to go. Except that I run my WebXR application using SSL, with a custom certificate (I have a DNS entry that points a local IP) and mixing secure and unsecured connections doesn't work in this case.

To get this to work you have to find where Vorlon is installed on your Windows machine. In my case it's %AppData%\npm\node_modules\vorlon\Server. You'll have to edit the config.json file there. Basically it comes down to changing "useSSL": true.
When you want to run SSL on LocalHost this is enough. In my case, I wanted to use my own certificate. To get this to work I copied the .crt and the .key files to the /cert folder. I also updated the config to use these. Don't forget to update the script tag in your .html file to use the domain name if that's different now. I added <script src="https://{YOUR-IP}:1337/vorlon.js"></script> to my HTML file.

After this, you just open the Vorlon URL (https://{YOUR-IP}:1337) on your desktop and the WebXR app on the HoloLens. Anything that is written to the console will show up on the desktop. But you can explore a lot more on the remote location from your desktop machine.

How is CSS applied to your document?

𝓢𝓸𝓻𝓼𝓴𝓸𝓸𝓽🌷 — Sun, 05 Jan 2020 23:39:13 +0000

This week I was debugging an issue with the CSS of the application we are building at a client. It turned out that the CSS was applied to the document in an order that I wasn't expecting. It's about time for me to learn how CSS is applied to HTML documents.

How and when is CSS loaded?

Before we have a look at how the CSS is applied to a page, the CSS needs to be loaded first. This happens in the following way.

1) First, the Browser loads an HTML file
2) The HTML file is converted to the Document Object Model (DOM)
3) The browser fetches resources, like CSS or JavaScript
4) The browser parses the CSS and applies the style
5) A render tree is created
6) Lastly, the page is displayed

There are two important ways that CSS is applied to a page, its cascading order and the specificity.

Cascading Order

The CSS that is applied to a page can come from different origins. The cascading order determines what origin to apply first.

There are 3 different origins the CSS can come from:

User-Agent; These style sheets come from the browser and can be real style sheets or hardcoded values.
Author; These are the most common style sheets, the style sheets that are added to a website.
User; It's possible for users to override the style sheets and customize the look.

The order in which the origins are applied is:

	Origin
1	User Agent	normal
2	User	normal
3	Author	normal
4	Animations
5	Author	!important
6	User	!important
7	User Agent	!important
8	Transitions

Note that later defined styles within the same origin win.

Another thing that you need to keep in mind is animations. The @keyframes rules work a little bit different. They don't cascade and don't mix together. If multiple @keyframes are defined, the last one is chosen.

Specificity

Within the cascading of origins, the specificity of the CSS is used.
The specificity is calculated by adding weights to the different declarations:

0 for *
1 for elements (h1, div) and pseudo-elements (:before, :after)
10 for classes (.something), attributes([some-attrib]) and pseudo-classes (:hover)
100 for IDs (#specific-ID)
1000 for inline styles (<div style="color:#ff0000">)

Of course in CSS these will be combined which means the values are added together.

A few examples:

If we have a look at the following CSS, a few things happen there:

The text on a div with the ID some-id will be red. This is because line 1 calculates to 101 (an ID and an Element). The second line only calculates to 100, the last one calculates to 11 (and element and an attribute).

Now assume we have an H1 on the page. Without a class, this will become purple. Because the latest defined rule wins. If we would add a class (some-class) the text becomes yellow, a class is more specific than the element.

I think I get it now :)

DNS lookup for local IP

𝓢𝓸𝓻𝓼𝓴𝓸𝓸𝓽🌷 — Fri, 20 Dec 2019 15:37:31 +0000

Sometimes when testing you want to have easy access to your machine. For example, when building and testing a Progressive Web App running on your development machine, you probably want to test it from your phone as well. In a lot of situations, it is sufficient to just type in the IP address of your dev machine (assuming it's on your local network) and browse it. But in the case of a PWA, you need to have the website running on HTTPS.

How?

I registered a short domain name, c0dr.nl, and got a wildcard SSL certificate for that. I used DigiCert, but LetsEncrypt will work just as fine. Now instead of setting up a host with storage and such, just configure the DNS settings. I added an A record with an easy name, for example: 'twitch'. For the IP address, you'll have to add your local address. Not the one you got from your ISP, but the one you got from your local DHCP or that you entered in the IP settings. In my case, it's 192.168.1.13. That happens to be the same address I used for testing my PWA in the first place.

What's left is to make sure that whatever you use to run your web app is using the certificate.

One last note. You don't have to use a wildcard certificate. But it will give you a little bit more flexibility when your IP address changes. You might work in different locations. At home and at the office. You can set up two different subdomains for that. If you only need one, use an '@' instead of a name in the A-record.

Twitch Stream Setup

𝓢𝓸𝓻𝓼𝓴𝓸𝓸𝓽🌷 — Mon, 25 Nov 2019 13:45:00 +0000

Whenever I have some spare time I get into coding on my projects. A few years ago I started live coding, on LiveCoding.tv at the time. There were some people streaming programming on Twitch at the time, but only game devs and that was not what I was doing. I tried switching over to Twitch, but the lack of growth of the channel and lack of motivation from my side led me to stop. Earlier this year I decided to pick it up again. This time only on Twitch, where the community of Live Coders is growing fast. I felt welcome in the community. And I got affiliated pretty quickly, thanks to the support of the community. If you are looking for people coding on Twitch, you should have a look at the LiveCoders team.

A lot is going on when running a stream. In this post, I'd like to give you a look backstage to what is going on during the streams.

Of course, there's audio, video and a view of my screen. But there's a lot more going on...

Hardware/Software

Let's have a look at the hardware I'm using for the live coding streams first.
Because I'm doing projects with VR and graphics for work as well besides web development, my employer (Centric) was kind enough to provide me with an Alienware 17 R5 laptop. This is perfect for 3D modeling and running the Oculus Rift. But has also enough power to encode video for a live stream. Where I used to have issues with my previous laptop, like dropping frames and such, now this is completely over. I will get into more detail about OBS later.

This is the list of hardware I'm using:

Alienware 17 R5
HP LA2405wg Monitor (old, but still works fine)
Das Keyboard Professional
Logitech MX Master 2S
Logitec 920c
Blue Yeti
Elgato Stream Deck
8 port Sitecom powered USB hub

Software-wise I'm using OBS to stream. I'm using VSCode (mostly) to code with the Synthwave'84 theme.
For the audio, I use VoiceMeeter Potato to route the audio and to have fine control over the volume of everything. I use SoundByte to play soundFX. The music I play in the background during the stream is Monstercat. I build a custom player that runs inside OBS and renders a visualization of the audio. I've also got a chatbot running, which I also build myself. Her name is Rosie, named after the maid in the TV series The Jetsons. At the moment I'm using StreamLabs for the alerts.

Audio

In the early days of my stream, I used a cheap headset. The quality of the audio was very bad. The only benefit was that I could take it and stream from pretty much anywhere. I have it replaced with a Blue Yeti. This is a UBS microphone that could be used on its stand. I have removed this stand and screwed it onto a boom arm. During a stream, it's just below the camera, between the keyboard and my mouth. When I started streaming I didn't bother too much with the audio. But slowly I'm starting to get more and more obsessed with it. I think it's one of the hardest things to get right when streaming. And I know that at one point in the future I will replace the Yeti with a good XLR mic and replace VoiceMeeter with a hardware mixer. These things cost a lot of money though.

VoiceMeeter

During a live stream, I have a couple of things going on that make sound. I want to have full control of what the volumes of each are. I used to have everything set in OBS, but that wasn't fine-grained enough. I needed more control. Then I came across VB-Audio VoiceMeeter. Since I wanted to have the most control possible I got the VoiceMeeter Potato. This tool is donationware, which means you can pay whatever couple of $ you want for a license.

I have my mic, the music, sound effects, and my desktop all going to separate channels. I also use a channel in VoiceMeeter for a USB mixer I use sometimes outside the streams. I also have a channel reserved for Spotify. I'm not using Spotify anymore during the stream but might listen to it while working.

Let me explain my VoiceMeeter set up in a little bit more detail. Here's a screenshot of my setup:

If we go over the channels from left to right, we start with the channel for the microphone. I've routed this to B3 only. The 'B' channels on the mixer are virtual outputs, where the 'A' channels are routed to real hardware outputs. I'm only using B3 for my microphone. This way I can get it onto a separate input in OBS and have OBS mute it.

The second channel is the output from OBS. I'm using an extra free program from VB-Audio here as well: VB-Cable. This program gives you an extra 'hardware' audio output to work with. On top of the 3, you get with VoiceMeeter Potato. I've routed this channel to B2 for use in OBS and A1. A1 I've routed to my Blue Yeti. The microphone has also a headphone output and volume control on there. Without routing the sound to here I won't hear anything but my voice.

I'm skipping the other hardware inputs since they are not used for streaming, and go to the virtual inputs. For the stream, I use only VoiceMeeter AUX for my desktop audio and VoiceMeeter VAIO 3 for the sound fx. Both are routed to B2 and at least A1

Here's a sketch of the setup.

As an experiment, I tried having the audio on channel 2 going only to stream (B2) and Spotify only to my headphones (A1). To my surprise this actually works :) This way I can listen to some music while the viewers of the live stream listen to something else. I'm not planning on using this while streaming, but it is nice to know that it is possible to do things like that.

If you have trouble getting the outputs of your applications routed to the right channel in VoiceMeeter. Try going to the Windows Mixer, by typing mixer in your start menu.

Within these settings, you can specify which output should be used for every app. I have set SoundByte to output to VoiceMeeter VAIO3 Input and OBS to CABLE Input.

Normally you don't need to route OBS to a separate channel. If you are using alerts from StreamLabs with sound you might, but you'll probably be fine without. I wanted to do something special with the audio. So, I created a music player to play and visualize the audio myself. Since this is running inside a browser window in OBS, I used VoiceMeeter to control the volume.

Audio inputs and routing in a list:

input	from	routed
Blue Yeti Microphone	Hardware In	B3
OBS (music)	VB-Cable	A1, B2
Desktop Audio	VoiceMeeter AUX	A1, B2
Sound FX	VoiceMeeter VAIO 3	A1, B2

OBS

Inside OBS I don't have to do a lot with the audio anymore. I get the correct mix and the mic on a separate channel. I added them to two different input channels in the Audio Settings in OBS.

Instead of using the VoiceMeeter names I renamed them in the mixer so it's clear what both channels are. I've set the volume of the mic a little bit higher than the rest of the audio.

To make my voice a little bit better I've added a couple of filters to the mic. In OBS you can use VST Plugins. These are plugins that are very common in audio programs. I'm using the free Reaper VST plugins to improve the audio. The settings are not perfected yet and I'm constantly improving them to create a better sound.

If you don't care too much, you only need the first. This is a noise gate. A noise gate creates a minimum level of audio to be used. If the audio is below this threshold the audio is muted. This removes any noise the mic might pick up when you are not talking. I've used the one that came with OBS. You'll have to play a little bit with the settings to find the settings that are right for you.

Noise Suppression is the second most important filter I use. This filter removes a lot of the noise at the times you are talking. There's always a lot of background noise coming from my PC. Noise suppression takes care of that. In this case, I'm using the ReaFir VST. You can train this to create a noise profile and use that to remove it while talking. I've also created a second noise suppression profile for when my fan is blowing. I'd rather not use this one since there's a lot of suppression going, which affects the sound a lot. These are the settings I use normally:

Compression is used to balance the louder and quieter moments while speaking. This makes sure the audio doesn't get distorted when talking too loud, while also boosting a bit when talking softly.

The last filter I use EQ. This filter is used to boost or suppress frequencies. Both the compressor and EQ filters are in constant motion, I tweak these a lot during streams.

There are a lot of tutorials on these VSTs available on YouTube. Like this one from Tuts+ Music & Audio.

Video

The first purchases I made for the stream, I think even before my first stream, was a webcam.
From the start, I've been using a Logitech C920 Webcam. The quality of this camera is pretty good, for its price. It is very easy to set up. Just plug in the USB and you are good to go.

In OBS I have 1 webcam source I use everywhere. It is a bit laggy when started and I don't want it to restart when switching scenes.

For settings. On the webcam settings, I disabled all of the auto adjustments. I don't want anything to change outside of my control. I'm not moving around so the focus doesn't have to change. And I have people control my lighting through chat, therefore I want to keep white balance and exposure always the same. The only problem I'm having with these settings is that they are reset now and then.

In OBS I've added a couple of filters on the camera as well.

The only one that is making a real difference is color correction. You'll have to play with these settings yourself to see what you like.

I think that without color correction the video seems a bit too gray.

The other two filters I use are a crop for changing the width of the video a bit to make it better fit my layouts and I also add a little tiny bit of sharpness (0.05).

Lights

At the moment I'm using 3 lights during my stream. 2 of them can be controlled by the viewers by giving a !light command in the chat.

I use these lights:

I've got the white IKEA light pretty close to the left of me. The Neewer is a bit further away and pointed towards the wall to give a more diffuse light. The colored one is behind me, just outside the camera view and illuminates the back wall.

I bought the IKEA light just to try them and see how they look. I've got a couple of Phillips Hue lights around the house as well. The great thing about the IKEA smart lights is that they connect to the base station of the Hue lights and can be controlled in the same way. If you have a Hue bridge you don't need to buy anything else but the light bulbs to use those.

OBS

The application I use from the stream itself is Open Broadcast Software or OBS. This program lets you create scenes, configure what you want to capture and cast it to various sources, like Twitch or YouTube.

I've never used any other program for streaming. I'm using it to stream to Twitch, but I've also used it to stream to YouTube or record videos. You can have different setups stored and it is easy to switch between them.

Scenes

OBS makes use of scenes. Scenes are what you see when watching the stream. They build out of various sources like the webcam, desktop capture, animations, texts, and browser windows.

I recently cleaned up the scenes and the sources. It was a mess with all kinds of old, unused and hidden sources. I created 2 scenes that are reused in various other scenes, alerts, and texts. I also added color so I can quickly see what sources are where in the seen. To be sure I don't accidentally move a source, I locked everything.

Pre stream

A few minutes before I go live I already start the stream. I send a tweet out at the same time. At this moment my followers are informed that I went live. Having a count down or waiting room gives everyone a few minutes to come in before I start.

I don't always use the timer. If it's very short or a weird number of minutes before I go live I hide the timer. I added a ticker that shows random texts, just for fun. This is a browser window that's coming from my layouts application.

The chat is an overlay coming from StreamLabs. I styled it to look similar to the theme I'm using in VSCode.

Pre stream - Webcam

When I start the stream I welcome everyone to the stream using this scene. It has chat and a big webcam view.
I kept the 'almost there' text in there so when people pass by the stream the know I'm about to start.

Regular Stream

This scene is what is used most during the stream. It has chat, webcam and a view of the desktop. The background animation in this scene has a mask and is actually on top of the webcam and the desktop view. I want to cut off my Windows taskbar and the easiest way of doing that is just hiding it. I tried using a transparent animation but that was way too CPU intensive, so I used the same MP4 but added a mask filter to it with a black and white image.

This scene has everything else going as well, the exploding emotes, alerts and the music player.

Regular Stream - Webcam

This scene is the same as the previous one but has the webcam and desktop views switched.

Be Right Back

Sometimes I'm interrupted and need to leave the computer for a few minutes. It rarely happens, but when it does I use the Be Right Back scene. I've got the chat and the alerts there.

Post Stream - Webcam

When the stream ends I switch to this scene. It has a big webcam view and the end credits to the side. The end credits are coming from my layouts. It doesn't have a chat, but still has alerts.

Post Stream

The last scene is similar to the previous but without the webcam. Sometimes I want to have the stream running a little bit longer, for example when I raid someone. When the raid happens I switch to this scene. The raid is not recorded by Twitch, but the normal stream is. This way you won't see my moving around but still have a few seconds extra when watching the VOD.

Control

StreamDeck

One of the most used pieces of hardware among streamers is the Elgato Stream Deck. This is a device with programmable buttons. Each button is a small display that can show information. The device is very powerful. I can't live without it anymore.

The Stream Deck has a couple of great features I use very often. The first one is the ability to trigger multiple things with 1 button, a Multi-Action. I use this for example to start all applications I need for streaming, or to change a scene in OBS and mute the Mic at the same time. I also have a couple of Multi Actions that I use when I go live. These send out tweets, set the title of the stream on Twitch, select the right scene in OBS and trigger OBS to start the stream.

Bot & Tools

To have a little automated help (and fun) I created my personal stream maid, Rosie the Chatbot in Node.js. She is inspired by the maid in the old cartoon The Jetsons. During the steam and in between I add commands and features to this bot.

To give an example, I created the !light commands. These commands use the Phillips Hue API on my local network to change the color of the light behind me. This light is also triggered when events happen during the stream.

Also, the sound effects are a lot of fun. These use Midi notes to trigger the effects in SoundByte. I also use Midi to lower the volume of the music playing during events.

I integrated Rosie with the Microsoft QnA platform. When a question is asked in chat, Rosie does a call to this service to see if there's an answer to frequently asked questions. For example what theme I'm using in VSCode or when my next stream is.

During the stream, I also run another Node.js application that is responsible for the overlays in OBS. The exploding emotes and even the music is run from here. For the music, I create a player without controls that just plays a random song from a folder. I have another page that is connected through WebSockets to control the music. I use the web audio API to create the visuals of the audio.

Closing

I think that's about it. If I forgot something I'll add it. Feel free to ask any questions about the setup, Rosie or layouts during my streams. I'm happy to help. So come and visit me at twitch.tv/sorskoot or join the discord.

🎃 Halloween Scream Stream👻: Making of

𝓢𝓸𝓻𝓼𝓴𝓸𝓸𝓽🌷 — Wed, 06 Nov 2019 14:43:36 +0000

This week was Halloween 🎃 and I wanted to do something special for the stream this time. I wanted to make everything a bit more spooky. So, I added some simple effects to the stream, created some spooky music and eventually added a few new shaders and wrote a horror script to end the stream with. This is what was added and how it was done. I hope this post will give you some ideas for your own streams. I might create some detailed tutorials on how to create the effects used in the future.

Look and feel

For the look and feel of the Halloween stream, I wanted to create an effect inspired by the dark world called the upside down from the Stranger Things series.

Overlays

I wanted to keep my overlays the same as normal as much as possible, with only a few added animations. I went searching and found a few animated gifs of smoke and floating particles I liked. Unfortunately, these had a black background instead of a green one. A green background can be removed pretty well using a chroma key filter in OBS. But, it turned out a black background can be removed as well. And since these gifs were black and white, the white parts stayed which gave me the look I wanted.

To make the combination of animations reusable, I created a new scene in OBS. and added the animations in there.

Then, in every other scene, I wanted to have the smoke and particles, I could just add a scene source and reference that scene. This way, whenever I need to make changes to the Halloween scene, I could just change it in one place and everywhere it was used the reference would update.

At some point, I noticed a higher than usual CPU usage in OBS. It turned out to be the chroma keying of the particle animation. I had to remove it to prevent possible issues while streaming.

Camera

In normal streams, I've configured the colors of my camera to be vivid with the default lighting set to blue and purple. This didn't fit the theme for Halloween and wasn't spooky enough.

I changed two things with regards to the camera. First, I had only the lights in front of me turned on. I made sure the lights stayed this way, I disabled the light commands in the chat (you can type !light with a color to change the color name during the streams). I place the main light on the floor next to me angled upwards. The other one is set to light my face a little bit from the other side.

Second, I added a color LUT (lookup table) filter to the camera. Normally this is used to change the colors a bit and make the color pop just a little more, but in this case, I dramatically changed it and added a lot more blues while lowering the saturation. This created the look I wanted for the camera.

Music

My main choice for music is the songs from the Monstercat library. But these didn't fit the horror feel I wanted to have for my stream. I wanted to have very slow and long droning sounds. I remembered a tool call Paul Stretch. This tool can stretch audio into the extreme. I took a couple of famous horror movie themes, like the theme from the movie Halloween and made it about 20 times as long. This resulted in a couple of songs that were over 1 hour long. To finish them I added a bit of EQ and normalization to them so they all sounded similar.

The script

Now that I had the look and feel down I wanted to do something special. I wanted to 'tell a story' during my stream, inspired by my favorite horror movies. I came up with a story of an old haunted house. Every 66 years, with Halloween, the ghost would come back haunting the family living in the house at that time, which would result in a lot of unfortunate deaths.

During the stream, I mentioned details of this story, like living in an old house and the big fire in the fall of 1953. The stream would start normal, except for the spooky atmosphere. I also mentioned the rain and thunder outside.

1 hour before the end of the stream I started adding 'events' to the stream, supernatural events. It started with knocking sounds and children crying. I left my screen a few times with the camera running. With me gone or looking the other way, the camera glitches. The door would open by itself and when I'm gone to check it out and close it a black shadow moves past the camera.

Near the end, the light flickers and a ghost is seen in the background after which I decide to end the stream and check on the family because I keep hearing noises. Again, knocking. So I stand up from my desk and the stream starts glitching and after a bone-chilling scream the stream cuts out.

Shaders

A month ago I create a shader for use in OBS. I decided to do the same again and create a couple of different shaders. I used a tool called KodeLife to create the shaders. I've added 2 custom shaders and used 1 that came with the OBS shader plugin. Both shaders I created are not that complex.

I wanted to create a shader that would separate the RGB colors as you would sometime see on old CRT TVs.

This clip from the stream shows the effect around the 15s mark.

Twich Clip of RGB Effect

The shader is activated only when a semi-random value gets above a certain threshold. At that point, the Red channel shifts a bit to the left and the Blue channel a bit to the right. Below is the shader itself.

The other shader I created renders noise and randomizes when it is shown and how much. Again with a certain threshold and a specified speed.

The ghost

The ghost appears two times during the stream. The first time it passes the bottom of the screen as a dark shadow. This was created by overlaying a transparent animation on top of the camera.

The second appearance was a bit more work. To create the effect of the ghost appearing behind me, I took a screenshot of the webcam without the lights on, but with me looking into the camera. I opened it in Photoshop and added a figure in the back. Looking at the picture itself it looks too fake.

But, when I added the image to a scene in OBS (behind the camera) and added a flickering effect to the webcam on the alpha channel you just got to see it few milliseconds at a time.

Twich Clip of Ghost Appearing

I added a sound effect as well. It really looks like there's some glitch in the lighting. By this time during the stream, I was actually getting the chills, because of the weird lighting and the droning sounds.

Timing the events

For the timing of everything that was happening during the stream, I created a custom tool that was running in the background. This was a very simple web application that would show me a message and execute a function after a specific time had passed.

The tool reminded me to mention the 'storm' outside and the question of the day. I wanted to talk about horror movies during the stream a bit to try and influence the minds of my viewers. At some point,it started playing audio of knocking and children crying. It should be a message that it played the sound and reminded me what to do. At first, only listen, but later switch to certain scenes to show the glitches.

The entire code of this app is available at the Halloween Stream Repo on my Github.

I might reuse this application and add it to a bigger application I'm planning to create reminders and such for during my stream.

I also needed to automate a few things in OBS, because I didn't want to trigger things noticeably. There's a plugin for OBS that can help, Advanced Scene Switcher. There's an option in this plugin that lets you create sequences, it automates going to a specific scene after some time.

img-link "/images/2019/11/Horror-scenes.png" "list of horror scenes in OBS"

I wanted to have a little bit of time before a scene or glitch happened so I added 'pre'-scene to a glitch and timed it to the sequence. This way I could switch to a fullscreen camera, leave and then have a glitch occur. I also used this in the end. Right as I stand up I hit the last button, I stand up and grab my headphones. This way I created the illusion I didn't hit any buttons when the final malfunction happened because my hands are clearly visible.

I programmed everything into my stream deck and the special tool reminded me what to do.

Wrap up

I had a lot of fun creating the Halloween stream, even though a few things went differently than planned. I think I created the atmosphere I wanted, I actually was on edge during the stream. I did, however, forgot my lines during the end sequence and forgot to mention there was someone at the door again.

If you like to watch the entire ending you can do so in this highlight.