In my October diaries I mentioned helping friends to build a chicken coop. At that time my friend mentioned that it would be fun to have a web-camera live-streaming what the hen are doing. I said I think I can set up such a system for him. Well, in mid-December I asked for for confirmation if he is still serious about the web camera. "Yes" he said. He is a performing musician, and for the year 2019 they are going to have a busy tour schedule - a live video stream would be fun to watch while sitting in the band bus travelling from a festival to another. Sounds good! So I promised to provide the techical side needed for such a live stream. This blog post is about the process, so there will be quite a lot of programming jargon and technical details in the post. But I do my best to write it in such a way that there is something interesting for non-programmers, too.

Internet is full of videos, and some of them are live streamed - so nothing special there, should be a doable thing, yes. So what does it take to set up such a system? A) A camera capturing video content, B) the video being broadcast over the internet, and C) the video being displayed for the user. All of these can be further divided into smaller sub-tasks, until the whole task becomes a series of small little tasks which can be easily completed one after one. That was what I thought. To make it an interesting hobby project and an opportunity to learn new things I decided to use a Raspberry Pi for the step A. For the step C I was thinking about a web-page hosted on a server of some webhotel company. Based on my previous experiece I thought that in such a setup the web-page needs a way to communicate with the Raspberry Pi. A bit simplified, the problem in that is that internet has name servers which map textual addresses like 'www.enormouselk.com' to a number format internally used by the net communication protocols - but a single consumer device connected to a net does not have any name assigned to it. And most of the time, with most of the Internet Service Providers it is so that every time you connect your device to the internet your device gets assigned a different number which is its temporary address for that session. (Also, if you have multiple decices connected to a same (W)LAN router, they all share the same number as an address, and the router takes care of routing the net traffic to each device. But that router devices have a system called port forwarding which can be used to differentiate individual devices connected to the router.) To simplify: If I have a code running a website with a video player, and then the webpage wants to send a signal 'hey, start sending me a video stream' to the Raspberry Pi with a camera, then the website code needs to know a number-based address where to send that message. How to obtain that number, if it can randomly vary from a day to another? Well, but it is easier to the other direction; since the webpage has a static address assigned to it, the Raspberry Pi always knows where to find the webpage code. So I wrote small pieces of code; on the Raspberry Pi side there is a script which sends a simple "Hey I'm here!" message to the main webpage, once in fifteen minutes. When the webpage code receives that message, it can find out the address where the message came from, and writes that info in a text file. Now, when the webpage code wants to send a message back to the Raspberry Pi it can just check that text file to obtain the address. Basically, this is how DynDNS works, and there are a lot of services providing this functionality. I just wanted to write my own little version, just to better learn how it works.

I bought a camere module for the Raspberry and started experimenting with it. I quickly discovered that steps A and C can't be completely isolated, for there are things like video codecs and file formats. The official Raspberry Pi camera libraries support mjpeg and h264 -formats. I understand very little of the techical details of these things, but the difference seems to be that h264 is more efficienty compressed, which meas smaller files sizes so that it is faster to transfer over the internet. But, if we want an web-page and an internet browser for the step C the problem is that major web browsers don't support h264 video as such, but they wanted it packaced in mp4 container. Searching the net I found a lot of examples where people used a video player like VLC to play the live stream provided by a Raspberry Pi. That would be one option, yes, but I was interested in having the live stream displayed on a web page. That way one could access the live stream anywhere, post the link for friends to watch, and the web page could handle multiple clients watching the stream at the same time.

Well, but to get started I was following a tutorial on the official documentation. The example was written using a programming language called Python. I have only a minimal experience in Python, so I don't know the language very well. Also, I've been a bit lazy with setting up my current development environment; to write code for the Raspberry, I use command line teriminal instead of any modern IDE. And the terminal I'm using does not know how to handle stuff like copy & paste. So I found myself reading a source code which I didn't fully understand, and manually typing it in line by line, then running the script to see what it does. That made me feel like back in the 1980's, for that was the way I learned to program back in those days. (Of course then it was not google and webpages, but a magazine printed on paper.) I don't know but I feel that manually copying code line by line is better for learning than the modern "paint with a mouse, click to copy and paste". Or, at least, I somehow enjoyed the nostalgic feeling of being back in the learning patterns of my early teenage years. (Also, Python as a programmin language somewhat resembles the Commodore-64 language BASIC which was the first programming language I learned.) I got the code working - instead of a video stream it used a jpg-image, constantly updating the content of the image as the camera stream went on. That worked fine inside my home LAN, but seen from the outside it was laggy - only a frame or two per second. Apparently the bottle neck was the upload speed of my internet connection. That sent me back to considering the image format. Since h264 is more efficient, with the same upload speed it could get more frames transferred per second.

So, how to connect a h264 video stream to a web browser which would want the h264 wrapped in mp4? Generally speaking, I saw two possible approaches: Either convert the h264 to a proper mp4, or find a web-based videoplayer which can handle raw h264. Searching the internet I found various projects for each of these approaches. Also, that got me reading and learning more about video formats, transfer protocols and such. The more I searched the net the more I found blog posts, github projects, articles and forum threads about these techiques. I tried some of them - even those which required installing node.js on the Raspberry Pi. That didn't work, but I don't know why. For example, I don't know if I should've made sure that I installed the right version of node.js - maybe the code was written for some older version and I installed a newer version which was not fully backwards compatible? Oh so complicated! Also, I took a look at the source codes written for node.js - for the most part I could understand it, but since I have zero experience with node.js the overall project structure and all the details felt like too much to learn quickly. So I abandoned that route. Luckily enough, packaging h264 in a mp4 container is not so very complicated, and using a video format tool the Raspberry Pi could handle it reasonably quickly. Finally I had the Raspberry producing a video file that any modern web browser can play just like that. Progress!

But hey, a video clip is a different thing than a continuous video stream. I mean, I had used this kind of steps: 1) capture 10 seconds of video in h264, and save it to a local file. 2) make that file into a proper mp4-file, 3) display that file in a web-browser video player. But how can I achieve the step 2 if it is not a file with a beginning and an end, but a continuous stream of video data? Also, can I assign a video stream as a source for a browser-based player? After some further studies I learned that for video streaming we use something called 'a fragmented mp4' - which is basically a long video with bookmark-like notes inserted on regular intervals. If I undestood it correctly, those bookmarks can act as 'beginnings and ends' which the browser based video player wants to have. And that the video format tool can handle a continuous stream just like it can handle a file. I tried that, and it worked - only that the framerate was cut down to 6 frames per second. Now the bottle neck was the Raspberry Pi processor which isn't that fast (also, the vido fromat tool doesn't know how to use Hardware Acceleration). Again it felt like being back to my early days of learning computer programming - being restricted by the computer processor speed, thus needing to figure out alternative solutions which the processor could handle fast enough. Now, at the moment of writing this I am not able to recall if that was the third or fourt day of my coding project. On each coding session I slipped into a fully focused mode where I forget the passing of the time, I forgot to eat, I forgot about bed times, constantly just enjoying the cognitive challenge, the thrill of learning new things, the sense of adventure like 'but I want to see what lies behind that corner, so I keep on exploring further', until it was 2.30 am and I was hungry and sleep-deprived. Just like the golden days of my teenage years, when I could forget the daily reality of my life and escape into the abstract depths of lines of MOS 6510 assembler code (MOS 6510 is the Commodore-64 CPU, and when I learned its lean and efficient assembly code I quickly abandoned BASIC). And for a short while the rational part of my mind whispered a question: "Does all of this make any sense? Why I'm sinking all these nights into a hobby coding project, when I could just buy a ready-made commercial web-streaming camera and help my friend to install it?" But the answer left my rational mind satisfied; "Well, but with a ready project we are trapped with the way it works and we can't tailor it to suit our exact needs. Also, with a custom built system each component can be replaced independently if something gets broken, so the lifetime maintenace costs are going to be way less than with a ready-made product. On top of that this is great fun so please let me kindly enjoy what I am doing!"

After a lot of trial and error I decided to try this kind of approach: 1) Have the Raspberry Pi record a series of video clips, 10 seconds each (and keep on recoding under the client says it doesn't need the video any more). 2) Show those clips in a browser video player, one after one, quickly and automatically. Step 1) was relatively simple, for that I wrote a modified and extended python script. Step 2) took me to the more conventional web development. I used JavaScript to produce two video players. At first the player1 is shown and player2 is hidden - they are both positioned at the same place. When the player1 starts to play clip1, the player 2 is told to have clip2 as its source. When player1 is finished playing it is hidden and player2 made visible. Then player1 is told to have clip3 as its source, and you probably guess what will happen next =) And it will keep looping like this. There was a barely noticeable flicker when switching the players, and I consider that acceptable. So, with this hack I had turned a series of short video clips into a continuous stream. To keep track of clips to be played I used a separate text file which stores a list of clip names. Now, this is a bit simplified version of how the official http-based video streaming protocols like MPEG-DASH and HSL work. When it dawned to me that I had accidentally just re-invented a wheel which others have already done probably way better than I can do, I spent a few hours trying to test if I could utilize those established standards. But again, it seemed like the bottle neck was the Raspberry Pi processor speed. I understood that if something works with a new Raspberry Pi model 3 might not necessarily work with my old Model 1, for model 3 is about ten times more efficient than the model 1.

Then, after all the browsing through blogs, forums and projects I discovered that the Raspberry Pi official camera documents and tutorials have a working example of live video streaming and displaying it on an external web page. Another lesson learnt! Just because search engines will quickly and easily provide you with a tons of articles and discussions, which all start with the idea that 'since there is no official solution, I came up with this kind of hack to get the thing done', it still might be a good idea to spend a while patiently reading through the official documentation! The chapter about live video streaming to web was added and updated less than a year ago. Which means that all those blogs and articles I were reading contained old and out-dated information. Also, the solution on the official tutorials was simple and elegant. On the Raspberry Pi side it used a Python script, and a video format tool to convert the video to old MPEG1-format. And on the server side it just needed a piece of JavaScript code to decode the MPEG1 format. Very clever! But, based on all the experience I had accumulated during my days of trial and error, I anticipated that the solution might not work with my old Raspberry Pi Model 1, so I decided to stick with my current approach. And, also, one of the ideas was that a custom-built web camera software allows future updates - something which wouldn't be so easy with a ready-made commercial product.

Friday, it was going to be a night of jazz music at one of my favourite bars at Tampere. My friend would be performing, too. For the first time playing pieces he had composed, together with his fellow musicians he had invited as an 'All Stars' band for this special occasion. The atmosphere was special - the place was almost fully packed, and everybody was paying full attention to the bands playing. And good jazz music, it touches your emotions and tingles your imaginations, sparkling with creativity. One of the songs my friend's All Starts band played was composed using a rather unusual method; they had a long sheet of blank staff paper placed on the floor. Then the rooster of their flock of hens got his feet cleaned and painted with fingerpaint. The rooster then started from the beginning of the stave, leaving coloured prints as he went on striding towards the other end. So the main composition work was given for the rooster to do. The band then arranged it into a full-blown jazz piece. Sure thing, I would be glad to provide a web camera so that my friend can always check what his jazz-composing rooster is doing in the chicken coop =)

Saturday. After a tasty breakfast and a good dose of strong black coffee I started to set up all the equipment needed for the web camera. I started with establishing an internet connection for the Raspberry Pi, logging into their home router and adjusting the settings for port forwarding. But for some reason it didn't quite work the way it did at my home. I spent a hour trying to figure out what was wrong, and I eventually learned that there is such a thing as Carrier-grade NAT (CGN). A bit simplified, it means that their home router sits behind a 'super router' at their Internet Service Provider's machine halls. And since I can't access the settings of that 'super router', my port forwading and dynamic DNS system would not work. Reading wikipedia articles about these things, they said that the downside of Carried-grade NAT is that it makes it impossible to use web cameras and other such devices which would want to serve web content to other users. So, the odds are that even in the case I had just bought a ready-made commercial solution, it would not work with they way their internet connection is configured at the ISP end. I asked for a can of beer and tried to think what to do now. The internet said that there are protocols and services to get around the limitations of Carrier-grade NAT, but somehow I felt that I don't have enough time to properly read and deploy some new techiques of which I just today heard for the first time. So I modified my existing system so that the code at the main web page does not need to send signals back to the Raspberry Pi. Instead I added another small text file on the webhosting side. That text file reads "STOPPED", but when an user clicks to start a video stream, "RUNNING" is written to the text file. On the Raspberry Pi side there is a script which is scheduled to run once a minute. The script fetches that text file from the webpage, and if the file reads "RUNNING", the Raspberry Pi starts to capture video clips, and pushes them to the webpage. So that the webpage does not need to ask "hey, I want to have a video clip from this spesified address which I fail to determine", it just needs to listen for incoming messages, and when it receives a file containing a video clip it saves it to the webhosting space, and adds a line to the playlist. Again, I was totally immersed in the coding, feeling that if I just keep on pushing, I will get everything up and running today as intended. I got to the point where we had videoclips succesfully transferred, and the webpage reliably displayed them, and it was working on Chrome, Firefox and also on iPhone Safari browser. Progress! The main functionality was there, now it only needed a little bit of additions to smoothly handle stream start and stop, and to catch exceptions and errors. And it lacks a system to hanlde such cases where one client is already wathcing a live video stream, and while the stream is still active an another client opens another browser to watch the stream - now, instead of starting over, we could just serve the already-running stream to the new client. I wanted to have all that done before declaring it ready. Then the lovely wife of my friend came to gently tell me that it is 10pm already. And there would be another jazz night at that nice bar, with world-class musicians performing. I knew my friend wanted to go see them, and I didn't want to be a burder. So I accepted my semi-defeat, semi-success. Instead of going to install the half-functional camera in their chicken coop I promised to return the other week when I have completed all the coding. I packed my stuff and we went to listen some good jazz music.

Well, but all the techical details aside; I've felt rather happy about being back to this kind of focus and enthusiasism. Ignoring some sleep-deprivation, just willing to learn and to code for the mere fun of it. And knowing that the hours I've spent alone forgetting about all the social world, being totally immersed into the abstract world of computer code - all of that will produce a connection of sharing between me and my friend, and my friend and their flock of hen - and the rooster who composes jazz music. That, seen from a broader perspective, the solitude of coding can be a way of being-together and sharing with friends, if you are coding something your friends are going to use in their daily lives. Something which is uniquely tailored to their needs and wishes. And then it was 2 am, I was walking alone on the quiet streets of Tampere, whistling to myself and occasionally taking a few dance steps to cross a street, for I felt happy.