Forem: Ivan Kuten

Making React Work on Smart TVs: Behind the Scenes of the Sportworld App

Ivan Kuten — Thu, 30 Oct 2025 14:53:02 +0000

TL;DR: Building apps for Smart TVs with React is nothing like the web. Here’s how we made a sports streaming platform stay stable and responsive on Samsung TVs — even with memory limits, performance quirks, and old hardware.

When we were asked to bring a sports streaming app to millions of Samsung Smart TVs — including models as old as 2017 — we knew performance and compatibility would be our biggest hurdles.

And yes, those hurdles were real. Even after months of tuning, older Samsung models still push the limits — memory leaks, dropped frames, unpredictable behavior. But the app holds up. It recovers, it doesn’t crash, and that’s a win when you’re running React on a TV from 2017.

About the Project

Our client, NativeWaves, delivers interactive streaming experiences for sports and entertainment. Together with B1 SmartTV, they launched Sportworld, a sports hub preinstalled on Samsung TVs in Germany.

Our task was to rebuild the app for Tizen OS, and make the product ready for future ports to WebOS and FireTV – while implementing new interactive features such as switching camera angles, jumping into live replays, and exploring on-demand content.

Eventually, the platform became part of a partnership with the German Bundesliga — the country’s top professional football league — giving fans access to live multi-angle match experiences.

Engineering Challenge

Developing for Smart TVs is a different world. We had to make sure the app:

Runs smoothly even on older Samsung devices (2017+ models) with limited memory and CPU power.
Provides a premium, lag-free video experience (live and on-demand).
Handles multiple interactive features — without breaking UX.

On top of that, we needed a modular architecture that would make porting to other TV and mobile platforms fast and predictable.

How We Tested and Profiled It

Profiling React apps on Smart TVs turned out to be a challenge of its own. To get accurate performance data, we had to use older versions of Chrome — version 79 for 2018 TVs, and for 2016–2017 models even Chrome 56 on a separate old machine.

Modern DevTools simply don’t show what’s really happening inside Tizen’s JavaScript engine.

We also tried the emulators from Tizen Studio, but success was hit-or-miss. They only work if the host CPU supports hardware virtualization, and there’s no ARM build — which means no luck on modern Mac M1+ machines.

In practice, debugging on real devices was the only reliable way to see how the app behaved. This experience shaped our approach: measure performance where it actually runs, not where it’s convenient.

Tech Stack and Architecture

We built the frontend with a lightweight and highly responsive setup:

React.js, TypeScript 
Styled-components 
Zustand 
Webpack

The Zustand state management turned out to be simpler and faster than Redux for our needs, while styled-components helped us dynamically theme different app sections (global and per-page themes).

We also put special effort into keeping the bundle small and efficient — avoiding unnecessary re-renders and using lazy loading for heavy components like video players and analytics modules.

The backend was provided by the client, with well-structured REST APIs documented via Swagger. This setup allowed our team to react quickly to API updates and maintain high development velocity.

Why React on Smart TVs Is Challenging

Running React on Smart TVs isn’t a matter of framework — it’s a matter of physics.

We didn’t choose React; it came with the project. Our job was to make it work inside the limits of Tizen and WebOS — where every render and memory allocation counts.

Even mid-range TVs have less CPU and RAM than most phones. Their browser engines are outdated, garbage collection is slow, and firmware updates are unpredictable. That means even small things — image size, runtime CSS, a few extra listeners — can affect playback smoothness.

We tuned what we could: cut bundle size, trimmed re-renders, and lazy-loaded heavy components. Locally, in the browser, performance looked great. On real TVs — still tight. The app stayed responsive and didn’t crash, but the margin for error was razor-thin.

What made it tricky

1. Hardware ceilings.
Older Samsung models have minimal CPU and GPU power. React’s re-renders can easily cause dropped frames or sluggish navigation. We profiled the UI, flattened component trees, and stripped unnecessary animations to keep interactions predictable during live playback.

2. Outdated JavaScript engines.
Tizen and WebOS browsers don’t match modern Chromium. Some ECMAScript features behave differently, and bundle size directly affects startup time. We tested every polyfill and adjusted TypeScript output for backward compatibility.

3. Remote-only UX.
No touch, no cursor — just arrow keys and OK. We built our own focus management layer to make remote navigation feel instant and natural.

4. Video meets React.
Multiple streams (live, replay, VOD) run alongside the UI. To prevent freezes, we isolated video logic from React’s render cycle and throttled event updates.

5. Debugging in the dark.
Smart TV SDKs give little visibility. Deploying to devices takes minutes, not seconds. We built emulators and detailed logs to reproduce TV-specific issues locally.

In the end, React on Smart TVs isn’t about libraries or syntax. It’s about patience and precision — squeezing stable performance from hardware that was never meant to run a modern frontend framework.

Key Features We Implemented

Live HLS streaming and VOD playback
Event calendar and promoted highlights
QR-based login and SSO integration via Plenigo
Paid subscriptions and campaign support for new TV models
Custom Google Analytics events for in-depth engagement tracking
Support for deep links (jumping directly to a live match or replay)
Optimized UI for remote control navigation (a major UX difference vs. mobile)

These features turned Sportworld into a fully interactive second-screen experience — where fans can easily switch camera angles, follow replays, and explore on-demand content.

Portability Matters

From the start, we structured the app with platform-agnostic modules and minimal dependencies on Tizen-specific APIs.

That decision paid off later: when the client wanted to expand to WebOS and FireTV, we could reuse about 80% of the codebase with minimal adaptation.

We also designed the architecture to support future integrations — from new sports to additional analytics providers — without rewriting core logic.

Lessons Learned

React on Smart TVs is all about restraint.
Performance tuning matters more than adding libraries. Profiling tools and manual optimization saved us from micro-stutters on older models.

Plan for portability early.
Even if you start with a single platform, build with modular architecture. It makes future ports dramatically faster.

UX for TVs ≠ UX for mobile.
Designing for remote navigation and slower hardware requires a different mindset — focus on responsiveness and minimal animation complexity.

Dedicated teams work.
Our setup — 3 engineers at start, scaling to 9 at peak — kept communication efficient and made delivery predictable for the client.

What’s Next

The Sportworld app is now pre-installed on Samsung TVs in Germany and continues to evolve with new sports and interactive features.

Also, we launched the WebOS version — built on the same core architecture. The only difference: it’s not pre-installed, but available as a downloadable app through LG’s content store.

This step confirmed that our platform-agnostic setup works in practice, not just on paper. The same base now serves as a foundation for future ports to FireTV and other devices.

I prepared this article together with the engineers from our dedicated team at Promwad, who built and optimized the app. If you’d like to see the demo video and more details from the project, check out the full Sportworld Smart TV App Case on our website.

Linux and CPSS Assembly on Marvell Board with Prestera DX Switch

Ivan Kuten — Mon, 04 Oct 2021 12:00:34 +0000

We've prepared a step-by-step guide for assembling and launching reference firmware with CPSS on the Marvell RD-AC3X-48G4X2XL-A board. This board serves for managed switches with up to 48 ports design and debugging. What makes it a perfect fit for an engineer? It has a Marvell Prestera DX 98DX3257 switch with a powerful Prestera SoC family on board. Besides, Prestera DX Ethernet switches enable 5G at the industry's lowest requirements for power and space.

As for CPSS, it is the basic software layer for Marvell Prestera hardware. CPSS stands forCore Prestera Software Suite. Open Compute Project uses CPSS as a part of the SONiC open network operating system. This OS extends data center functions in complex systems, such as intelligent data storage and machine learning.

Moreover, CPSS high-level API helps developers leverage features of Marvell Packet Processor without in-depth knowledge of Prestera chipset registers and tables.

CPSS locates in the user environment. It has several functions that require kernel-specific support:

Interrupts: a dedicated kernel module for forwarding Packet Processor interrupts to the CPSS.
To-and-fro CPU packet: another kernel module for forwarding packets between the PCIe interface to dedicated SDMA memory.

That API allows complete configuration of the Prestera-based switch.

Our board already has a u-boot. It is loaded into the device nand-memory on the vendor side.

Before following the steps in our guide, please, create an account on the Marvell portal.

Step 0: Create a working directory

mkdir ~/marvell && cd marvell

Below you’ll find all the components you need to download and unpack in the ~/marvell directory (~/marvell). All the paths in the guide are relative to it.

Let's start from the Products section of the Marvell portal:

Step 1: Download the toolchain (compiler, etc.) to build the kernel and cpss-application

1.1 Go to the Marvell portal and select Products in the top center.

1.2 Follow Tools→Marvell GNU Tools→MGCC-5→201511 on the left in the menu.

1.3 Select Toolkit in the window on the right and download the necessary toolchain. At the time of this guide preparation this is armv7 marvell linux gnueabihf hard 5.2.1 x86_64 20160301.tar.xz.

1.4 Unpack the archive to the working directory.

Step 2: Download the Marvell assembly system with CPSS sources

2.1 In the same Products section on the Marvell portal, select Switching→Software→CPSS-DXCH→4.2.2020.3 from the menu on the left.

2.2 Click on the window on the right side of Base SDK and download CPSS 4.2.2020.3 release – GIT Source.

2.3 Unpack the archive into the working directory.

Step 3: Download instructions for assembling and using the Linux Marvell and CPSS image

3.1 Again from the menu on the left, in the Products section, select Switching→Software→cpss→4.2.2020.3.

3.2 Click the User Guide in the window on the right and download PSDK PIPE 4.2.2020.3 Release – User Guide.

3.3 User Guide is a set of html-pages for browsers. To view it, unzip the archive and open the page in the root – PSDK_home.html.

Step 4: Clone the Linux-Marvell kernel from Marvell Github and configure it

4.1 git clone https://github.com/MarvellEmbeddedProcessors/linux-marvell.git

4.2 mv ./linux-marvell ./cpss_release_git_CPSS_4.2_2020_03_018/kernel

Please note that the kernel must be located near the sources of cpss. Its name must be kernel.

4.3 cd ./cpss_release_git_CPSS_4.2_2020_03_018/kernel

4.4 git checkout remotes/origin/linux-4.4.52-armada-17.10

4.5 export ARCH=arm && make mvebu_v7_lsp_defconfig

Next, enable RAM disk support in the kernel. There are two ways to do this:

– via make menuconfig

– or write the following code in your .config file:

CONFIG_BLK_DEV_RAM=y

CONFIG_BLK_DEV_RAM_COUNT=2 (maximum possible number of RAMs)

CONFIG_BLK_DEV_RAM_SIZE=524288 (the maximum size of a RAM disk in kb that the kernel can support; 524288 is 512 mb)

Step 5: Configure the desktop environment

Refer to the User Guide from Step 3 for the configuration process.

5.1 cd PSDK_PIPE_4.2.2020.3_UserGuide

5.2 Open PSDK_home.html.

5.3 CPSS User Guide on the left.

5.4 Building CPSS → Building CPSS in Linux → Installing the Development Environment — on the left.

5.5 Skip Tool Chain Installation, we have already installed it. Go to Configuring the Development Environment. There we make all further configurations.

Next steps (6-12) are taken in the ./cpss_release_git_CPSS_4.2_2020_03_018/cpss directory

Step 6: Build the kernel and the device tree

6.1 git checkout cpss_4.2

6.2 make ARCH=arm CROSS_COMPILE=$(pwd)/../../armv7-marvell-linux-gnueabihf-hard-5.2.1_x86_64_20160301/bin/arm-marvell-linux-gnueabihf- TARGET=armv7 FAMILY=DX DEBUG_INFO=Y kernel

Step 7: Assemble CPSS

make ARCH=arm CROSS_COMPILE=$(pwd)/../../armv7-marvell-linux-gnueabihf-hard-5.2.1_x86_64_20160301/bin/arm-marvell-linux-gnueabihf- TARGET=armv7 FAMILY=DX DEBUG_INFO=Y appDemo

Step 8: Build the kernel modules to make CPSS work

make ARCH=arm CROSS_COMPILE=$(pwd)/../../armv7-marvell-linux-gnueabihf-hard-5.2.1_x86_64_20160301/bin/arm-marvell-linux-gnueabihf- TARGET=armv7 FAMILY=DX DEBUG_INFO=Y modules

Marvell provides 3 kernel modules:

mvIntDrv.ko — for processing interrupts from/for Packet Processor.
mvDmaDrv.ko — o for allocating the DMA memory that sends packets to/from the CPU.
mvMbusDrv.ko — for accessing internal resources when using Prestera's internal processor.

Now, let’s assemble the file system:

Step 9: Download the busybox 1.21.1

mkdir busybox && wget -P busybox https://busybox.net/downloads/busybox-1.21.1.tar.bz2

Step 10: Fix the Makefile

Open the root Makefile in cpss_release_git_CPSS_4.2_2020_03_018/cpss

and replace the line:

tools/build/rootfs/crfs.sh $(BUILD_FOLDER)/rootfs $(CROSS_COMPILE) /nfs/pt/swdev/areas/readonly/swtools/devsources/root_fs/files/busybox/busybox-1.22.1.tar.bz2; \

to:

tools/build/rootfs/crfs.sh $(BUILD_FOLDER)/rootfs $(CROSS_COMPILE) $(MY_DIR)/busybox/busybox-1.21.1.tar.bz2; \

Step 11: Build the file system

Instead of storing on the device, the file system will be stored on a RAM disk. The RAM disk will be loaded into the device RAM. This is the most beneficial storage option, since the amount of memory used does not exceed 256 MB.

11.1 make ARCH=arm CROSS_COMPILE=$(pwd)/../../armv7-marvell-linux-gnueabihf-hard-5.2.1_x86_64_20160301/bin/arm-marvell-linux-gnueabihf- TARGET=armv7 FAMILY=DX DEBUG_INFO=Y rootfs

We have already built the CPSS and the module to work with it in steps 7 and 8. Now copy them into the file system:

11.2 cp compilation_root/cpss_4.2/armv7_DX/appDemo cpssEnabler/mainExtDrv/src/gtExtDrv/linuxNoKernelModule/drivers/mvDmaDrv.ko cpssEnabler/mainExtDrv/src/gtExtDrv/linuxNoKernelModule/drivers/mvIntDrv. ko cpssEnabler/mainExtDrv/src/gtExtDrv/linuxNoKernelModule/drivers/mvMbusDrv.ko compilation_root/cpss_4.2/armv7_DX/rootfs/home/user/

Compile a file system image that is suitable for u-boot. We do this with mke2fs and mkimage:

11.3 cd compilation_root/cpss_4.2/armv7_DX && mke2fs -L '' -N 0 -O ^64bit -d rootfs -t ext2 rootfs.ext2 256M && gzip rootfs.ext2 && mkimage -A arm -O linux -T ramdisk -C gzip -n "Build Root File System" -d rootfs.ext2.gz rootfs.ext2.gz.uboot && cd -

The kernel, device tree, and RAM disk of the file system will be booted to the device via TFTP. It is also possible to write them to the NAND-memory of the device after the end of the development and debugging cycle. But we will stop with TFTP. You will need a connection between the host machine and the device (Ethernet port on the back of the device) and a TFTP server on the host.

Step 12: Install and configure the TFTP server

12.1 sudo apt install -y tftpd-hpa

12.2 sudo systemctl enable tftpd-hpa && sudo systemctl restart tftpd-hpa

Copy the kernel, device tree, and RAM disk file system to /var/lib/tftpboot. /var/lib/tftpboot is the home directory for the TFTP server.

12.3 sudo cp ../kernel/arch/arm/boot/zImage ../kernel/arch/arm/boot/dts/armada-38x-interposer.dtb compilation_root/cpss_4.2/armv7_DX/rootfs.ext2.gz.uboot /var/lib/tftpboot

Step 13: Set up u-boot on the device

13.1 Plug the Ethernet cable into the MGMT port on the back of the unit – this is the TFTP connection.

13.2 Connect the RS232 (host) – RJ-45 (device) cable to the CONSOLE port – this is your console.

13.3 Start minicom to connect to the device console. Port settings: 115200 8N1.

13.4 Turn on the device. Immediately, the bootloader gives the message: "Hit any key to stop autoboot:". Press any key to enter the u-boot settings menu.

13.5 Perform the following commands one by one in the u-boot console:

setenv bootcmd "run bootcmd_ram ;"
setenv bootcmd_ram "run tftp_ramdisk; setenv bootargs $console root=/dev/ram0 rw; tftp $loadaddr $image_name; tftp $fdtaddr $fdtfile; bootz $loadaddr ${ramdisk_addr}:${initrd_size} $fdtaddr ;"
setenv tftp_ramdisk "tftp $ramdisk_addr $ramdisk_name;
setenv initrd_size ${filesize}"
setenv ramdisk_addr "0x05200000".
setenv ramdisk_name "rootfs.ext2.gz.uboot"
setenv console "ttyS0,115200"
setenv loadaddr "0x02000000"
setenv image_name "zImage"
setenv fdtaddr "0x1000000"
setenv fdtfile "armada-38x-interposer.dtb"
saveenv

13.6 Boot! :-)

Step 14: Ensure, that the CPSS application works on the device

Load the kernel modules:

14.1 cd /home/user && insmod mvDmaDrv.ko && insmod mvIntDrv.ko && insmod mvMbusDrv.ko

Run the CPSS application:

14.2 ./appDemo

This takes you to the console of the CPSS application. In the console we can configure the Ethernet ports on the switch and the entire Prestera system. To check the application workability, we will connect one of the switch front panel Ethernet ports (any port) to a free port on the host machine. After linking them together, ping the host machine with the following commands:

14.3: autoInitSystem — initialize the Prestera system – this command must be executed after each reboot of the switch.

14.4: Next, run the following commands one by one

configure
mac address 00:00:00:00:11:22
interface ethernet 0/35
ip address
force link up
end
traffic
protocol enable arp
protocol enable ping
ping

If the ping is successful, you will see the following:

<Console(traffic)# ping 192.168.15.3>

<Pinging 192.168.15.3 using src mac: 00:00:00:00:11:22 src ip:192.168.15.10>

<Ping to ip 192.168.15.3 successful.>

Congratulations! You've just configured Linux and CPSS on the Marvell RD-AC3X-48G4X2XL-A board with the Prestera DX Ethernet switch!

Picture: Industrial Ethernet switch 1Gb/10Gbps based on OS Linux, developed on the same family of network chips as the debugger (48 downlink ports + 4 uplink ports, operating range: -20–70°C)

This guide is helpful for both theory and practice. You can explore the capabilities of the Marvell Prestera switch (L2, CAPWAP, CNC, DCE, PHY, EEE, IP, NAT, etc.) together with the Lua CLI or develop your own high-level Marvell Prestera-based application for network equipment.

A full description of the CPSS commands you can find in the User guide from Step 3 in the Lua CLI User Guide.

P.S. Please, feel free to follow me here or on my LinkedIn profile where I share engineering and business articles on the electronics design and manufacturing.

How to Develop & Publish Tizen Apps for Samsung Smart TV: Our Guide for Javascript Engineers

Ivan Kuten — Thu, 08 Jul 2021 08:42:47 +0000

This article is about the amazing adventures of JS developers in the world of Tizen OS. It will include a step-by-step setup of the integrated environment (IDE), development of Smart TV apps, their submission to the Tizen Store — and, of course, code samples and case studies.

If you share our belief that Javascript is the best programming language working almost everywhere, from your grandma’s Smart TVs to the SpaceX ships, welcome to our guide.
So, you’re a front-end developer having a big, beautiful, 65-inch Samsung Tizen TV. And you have it for a reason because you need to design a TV application based on Javascript: with an HLS player or DASH streams from some vendors, a bit of DRM, and other not yet clear acronyms. Nothing complicated, nothing personal.

There’s no room for retreat; you have to deal with it. Let’s get to the bottom of this together with the Promwad engineering team.

Installing Software and Setting up the Environment

The first thing to do is to go to the official website to get Tizen Studio. We are downloading this toolkit with particular attention to the Prerequisites section with system requirements and needed components.

Tizen Studio is an integrated development environment or IDE for Tizen that enables you to create real and virtual devices, connect platforms, create certificates and run applications.

Optionally, you can install the VSCode Extension for Tizen, it helps manage our IDE without leaving the editor.

Instructions for Mac on the Apple Silicon Platform

If you have a new ARM64-based Mac, you can install tizen-studio by doing the following:

right click on installer.app in Finder
select Show Package Contents
go to Contents/Resources/Java
unzip tizen-sdk.zip
copy the tizen-sdk/jdk folder to ~/.package-manager/
run installer.app

When the stars are aligned, once the installation is complete, you need to open Package Manager and add the following extensions:

Samsung Certificate Extension;
TV Extension Tools;
and the current platform (now it is TV-Extension-6.0).

Judging by the download speed, these packets are coming directly from the Korean servers. :-) While waiting for the installation to complete, you can read about the differences between HLS and DASH and between PlayReady and Widevine.

(…6 hours later)

Switching Your TV to Developer Mode

It’s time to move your TV to Developer Mode and add it to Tizen Studio.

To do this, go to Apps and press the magic sequence 1, 2, 3, 4, 5 on your remote control. As a result, the developer mode window opens.

If you have a full-featured remote with a numpad, you’re in luck. With a mini remote controller, it’s going to be a bit more tricky: you should open the on-screen numeric keypad (numpad) and press 1–2–3 buttons using the direction buttons (d-pad).

At the top: a full-featured remote control with a numpad and d-pad for advanced users and developers. At the bottom: a mini remote control for Smart TV with control buttons in the center.

In the pop-upped Developer Mode window, type in the IP of your device with Tizen Studio, reboot your Smart TV, and voila — you can open Device Manager and add your big and beautiful 65" TV.

Adding Your TV to IDE via Device Manager

Screenshot: Device Manager with a connected TV.

Several conditions must be met for a TV to be successfully added to Device Manager:

the TV is switched to Developer Mode;
the Developer Mode window shows the correct IP of your working machine with Tizen Studio installed;
devices are on a shared network: LAN, WiFi, or VPN.

[Optional] After adding a TV in Tizen Studio, you can copy its unique DUID, so it would be easier to create a certificate later. And if there are a lot of TVs (just in case), it is better to put all DUIDs into a text file in — one DUID per line.

Tizen Simulator and Emulator

The Tizen Emulator is almost a full-fledged replacement for a real device. It supports the emulation of processor, memory, peripherals, events, codecs, and graphics. However, there is no support for DRM or tech tools for digital rights management.

We advise you to download the 4.1.2 platform to run the emulator on it — it runs faster and takes up less memory space than others. You can find it in the archive section of the TV Extension for Tizen SDK.

A complete list of differences between the emulator and the real device can be found in the corresponding section on developer.tizen.org

Web Simulator is a lightweight version of the emulator in Chrome. There is virtually no API, and the rest is simulated with JS returning stub values. You can use it to test various screen resolutions and some basic stuff, but there are more limitations than opportunities.

Adding a Certificate on Your TV

Now it’s time to create certificates with the Certificate Manager. They will allow installing your application on your TV or emulator and publishing it on the Tizen Store. There are two types of certificates: Tizen and Samsung.

Screenshot: Samsung Certificate Manager.

Choose the Samsung certificate, it will allow you to install applications on your TV. To do this, you will need to create an account in Seller Office, a special section for Tizen developers on the Samsung website.

New accounts are confirmed manually, so you have to wait a bit — this is the main inconvenience of creating the first certificate. You will get the Public level by default, which is enough for most tasks.

The Partner level opens up more opportunities. To get it, you have to pass a quest in the Seller Office and wait for confirmation from two weeks to two months. (That was the case even before the COVID-19 pandemic!).

The Platform level is inaccessible to mere mortals.

The Tizen certificate is needed to test the application in the emulator ONLY. Its main advantage is your freedom. Freedom of choice: Public and Partner levels with its advanced API are available out of the box.

As a result, you will have a profile with two certificates — author and distributor.

Important: Take kare of your certificate (author.p12) and password, make a backup immediately, save it in a safe place, and send it to your client. Without it, you won’t be able to update the app in the Tizen Store.

Our tip for Tizen developers: before creating a certificate, enable your TV in Device Manager, then its DUID will be added to the distributor certificate automatically.

The Promwad’s profile with two certificates for Tizen OS: author and distributor.

To summarize. A certificate is needed for the following things:

wgt file signature to install on your device or emulator;
publishing your app on the Tizen Store;
certificate level defines a set of functions available from the device API.

Developing Web Apps on Tizen: Varieties and (Un)Usability

Tizen has two types of web applications: packaged and hosted.

A typical packaged application is installed on your TV and stores all resources locally — styles, scripts, templates, images, config.xml, etc. When developing or debugging such an application, you have to wait for the wgt file to be built, downloaded, and installed on the device. Writing styles, developing new modules, or debugging player behavior promises pain, frustration, and a great deal of time is lost in reconstructing the context.

The best solution in the early stages would be to develop an app for the Chrome desktop version. When preparing HTML layouts and debugging navigation, you can also do without a TV.

Lucky owners of the Samsung Partner level certificate can choose a hosted application. The TV acts as a resource loader, and the development process goes as the usual browser-based one. To work with hosted applications, you will need to do two things:

Add to the project your favorite http server with autoreload.
Create an empty project in Tizen Studio and add a redirect to the local machine IP to index.html.

<!DOCTYPE html>
<html><head>
<title>MyApp</title>
<meta http-equiv=“refresh” content=”0;url=http://YOUR_IP">
</head>
<body></body>
</html>

Done! The TV now acts as a webview.

The main disadvantage of such a solution is the reduced API of the webapis device. This thing is critical if you need, for example, to get DUID of your device to bind it with the back-end.

webapis.productinfo.getDuid()
Cannot read property 'getDuid' of undefined

Meanwhile, the Tizen API remains fully available: you can add RCU buttons, control volume, etc.

tizen.tvinputdevice.registerKey("ChannelUp");
success

What Code Is Used to Develop Apps for the Samsung Smart TVs

1. Javascript (ES5 and ES6)
In 2017, Tizen started partial support of the third coming of Javascript (ES6). Full support was implemented for the 2018 models. It is better to read the specification in person; and there may be surprises:

https://developer.samsung.com/smarttv/develop/specifications/web-engine-specifications.html#javascript-es6

For old-timers, you have to use vanilla.js or transpilate the code.

2. React
The answer is yes. We haven’t tried it, but a similar version is described on Stackoverflow: It’s a little challenging, but you can do it.

3. React Native
If you are a React Native developer, try ReNative.

Chromium and Webkit to run web apps on Tizen

Tizen has the Chromium engine to run web applications. This is true for 2017–2021+ TV models. The 2015 and 2016 models run apps on the Webkit engine and, despite obsolescence, there are still plenty of them in users’ homes. Gradually, Samsung is removing them from its app store. For example, today, you can no longer release a product for 2015 TVs.

The version of the engine does not shine like a new one and can not be updated. The 2021 model has Chromium M76 version released two years ago.

The reality is the following: to support current devices, you have to deal with the Chromium M47 version from back in 2015. What did you do in 2015? Ugh…

If that’s not enough, welcome to the support of 2016 TVs with the Webkit version of the mysterious r152340 revision. But let’s not talk about sad things.

Debugging Web Apps on Tizen

For debugging, you can use Chrome DevTools no older than version 79. That’s because on a chilly February morning in 2020, Google removed support for Custom Elements V0 starting with the 80 version, and those developer tools had since become incompatible with Tizen.

Workarounds like “Go to Preferences > Tizen Studio > Web > Chrome > Extra parameters” and add — enable-blink-features=CustomElementsV0 — user-data-dir=/tmp/tmpchromeuser do not work.

Our choice is to install the 79th version from the archive www.slimjet.com/chrome and disable auto-update.

For a TV from 2016 on Webkit, the DevTools from Chrome v62 or so will do. There is a high probability of getting a frozen screen on your TV if you press the “wrong” button in DevTools.

Publishing Your App: Seller Office and Tizen Store

It’s time to submit and publish your app for the Tizen Store. To do this, you need to prepare or request several things from your client:

1. Application logos according to the UX manuals:

A full-screen pic with two layers: the logo itself (transparent PNG) and the background. Size: 1920x1080px, 300 kilobytes max.
An icon for search results (512x423px), 300 kilobytes max.

2. Full-size screenshots of the main application screens — 4 pics.

3. A DOC file in English with a description of your screens and behavior of your application according to the following checklist:

UI structure displaying navigation between your app screens.
Use cases showing how to use the main features of your app.
Menus and functions — labeling and description of the main functions.
Remote control keys and their functions.
Language options.

Everything seems to be strict and clear in this official developer’s checklist, but as our practice shows, a simple document with UI screenshots and a description of your main functions and buttons is OK.

4. Specify the version of your application in config.xml and prepare a signed wgt file.

We’re good to go! The rest of the quest requires no prior preparation. You can go directly to the registration.

After successfully completing the registration, the manual testing process by Samsung begins. Their people add bugs to the internal bug tracker, attach silent videos there, and point out moments of your fiasco.

As a result, the application will appear in the Samsung Store, even if it has minor defects, such as “Many of channels don’t work. Black screen with loader is visible”. :-)

But with critical defects like “Infinite loading,” the application will not make it to users.

Screenshot: the bug tracking system on Samsung’s portal for Tizen developers.

After fixing the critical bugs, you can celebrate your success! The app can now be found by name or keywords in the Smart TV search.

Documentation for Tizen Apps Developers

The documentation is there, thank you very much.

It was written by different people at different times, so terminology and style changes from document to document, and in the guides have unworkable code. You can find a striking illustration in the example of using an embedded player: webapi instead of webapis. And the main thing: this unfortunate typo was in the DRM settings section! We are sure that we were not the only ones who reported this problem to the Sellers Office, but three years later the problem is still there.

Helpful Notes for Tizen developers

1. Registration and Support for Remote Control Buttons
The first thing you have to deal with when developing Tizen-based apps is registering the remote control buttons.

Some buttons will not be available in the app by default, and you need to register them first.

Individually:

tizen.tvinputdevice.registerKey('MediaPlayPause');

Or as a bunch:

const keys = ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", 
"ChannelUp", "ChannelDown", "MediaPlay", "MediaPause",
"MediaFastForward", "MediaRewind", "Exit"];

tizen.tvinputdevice.registerKeyBatch(keys, successCb, errorCb);

It is better not to register VolumeUp, VolumeDown and VolumeMute buttons, otherwise, their native functionality will require program processing via API.

The rest is easy. Catching clicks in the application with the Listener:

document.body.addEventListener('keydown', handleKeyDown);

where handleKeyDown will pass the button code.

2. Useful Tizen and Webapis Calls
A list of useful calls for the Tizen developers:

tizen.systeminfo.getCapability("http://tizen.org/feature/network.ethernet");
tizen.systeminfo.getCapability("http://tizen.org/system/model_name");
tizen.systeminfo.getPropertyValue("ETHERNET_NETWORK”, successCb, errorCb);
webapis.productinfo.getFirmware()
"T-KTM2DEUC-1202.3"
webapis.productinfo.getModel()
UNU7400"
webapis.productinfo.getVersion()
"3.0"
webapis.productinfo.getRealModel()
"UE43NU7400"
webapis.productinfo.getModelCode()
"18_KANTME_UHD"
webapis.productinfo.getLocalSet()
"EU_LITHUANIA"

Curious Cases We Encountered in Our Clients’ Projects at Promwad

Case Study #1: Voice Control Support (Korea vs. Others)
Voice control is completely helpless and ruthless at the same time. What it should look like: you say to your remote “TV, I want coffee” and 10 minutes later a courier delivers you a cappuccino. The manufacturer writes nicely about “you can register general voice commands, which trigger a listener when the user speaks them,” but behind every “can” is a “must”. As is usually the case, two troubles merged into one: the need to pre-register commands and poor voice recognition. And if you or your client gets the idea of adding voice control, be prepared to supplement the Volume Up and Louder dictionaries not only with synonyms but also with the similar-sounding Bloom Up and Laura.

Everything is much better in the Korean TV interface. But, as their folk proverb says, “How can you wag a tail if you don’t have one?”

Case Study #2: Quick Transition Inside the App and Deep Link
The prize in the “least used feature” category goes to Smart Hub Preview, which gives you direct access to the internal sections of the app. With this kind of linking, you can feature content or send your users directly to some favorite section.

The scheme shows an example of featured content.

Case Study #3: Mpd Stream Bug With 50 Days
When porting one project to Tizen 4.0 for the 2017 model, we had a problem: the sound did not work when playing live broadcasts (MPD streaming). The Volume Up/Down/Mute buttons and the sound control through API did not help. On other platforms — for example, LG Smart TV (WebOS) — there was no problem. The link to the MPD stream and the file content also did not cause problems — all tags and directives were supported by Tizen.

We had sound in our test streams.

Our doubts were related only to the availabilityStartTime directive, which specifies a zero point on the timeline and is defaulted to Unix epoch, i.e. 1970–01–01T00:00:00Z.

It turned out that in streams with the value of availabilityStartTime greater than 49.7 days, the sound was absent or disappeared within the playback process. And it made sense because 49.7 days or 2,147,483,647 milliseconds or 2**32 equals the maximum positive integer value in 32-bit systems (Number.MAX_SAFE_INTEGER). Simply put, there was just a bug in the Tizen multimedia stack, and there was nothing we could do about it.

How was it solved? The next day the TV automatically updated the firmware to version 1251, and the problem disappeared.

Conclusion

Tizen holds a solid market share, the platform is developing, new versions of TVs and software are coming out. At the same time, development does not become easier or more comfortable and is full of frustration. The attempt to create a single platform for all devices at once is ambiguous.

The biggest problem is documentation and usability. It covers all versions of all devices at once — web and native apps — and is full of bugs and flaws.

The developer.tizen.org website loads slowly enough to feel this (3–8 seconds), and it hasn’t changed in recent years. The search is so awkward that it is more convenient to google something and go to the main site, but it is important to make sure that the article or guide refers exactly to your TV and to the web version of the application. And the automatic switching of the site to its Korean version is a common thing.

The forums for Tizen app developers are filled with typical problems. Answers with outdated links lead to “pagenotfound” 99% of the time, and communication with the Seller Office is useless 100% of the time. You will most likely wait three weeks to get an answer that is irrelevant to the question. And that’s for 30% of your profits if you release a paid application.

Testing feels a lot like the lottery. To build a working player for video streaming, let alone DRM or advertising support is a challenging task for advanced engineers. :-) It’s not for nothing that Tizen promotes the paid 3rd party NexPlayer as “the only video player recommended by Samsung” — you should use it.

We suspect that some paragraphs of our guide may have caused your frustration and discourage you from Tizen. Nevertheless, if things were simple, this article would not have been born, so we’ll wrap it up on a positive note: Tizen is globally ranked first among all operating systems for Smart TV. Last year, this OS covered 13% of the global market and 30% in the UK. Judging by the recent forecasts, Samsung is not going to lose its position.

Even in the U.S. and Canada, where the leader board of Smart TV OS differs from all other regions (they prefer Roku there), Tizen is still in the top three along with Amazon Fire TV. So if you or your customers are planning to release apps for a wide international audience, Tizen is your must-have, despite all the pain and tears for front-end developers who write JS code for this operating system. At least we at Promwad Electronics Design House can confirm from our experience that with each newly published application for Tizen Store, your next one will be easier and faster to release

The Korean proverb says “No gains without pains” (노력 없이는 이득도 없다). Thanks for reading!

About the Promwad Blog

Hi, my name is Ivan Kuten. I’m a co-owner & CTO at Promwad, where we design and manufacture for our clients new electronics and embedded software for smart TV, telecom, automotive, and other industries. Feel free to add me on LinkedIn; I’d love to connect with you and share my experience.

How to develop Gstreamer-based video conferencing apps for RDK & Linux set-top boxes

Ivan Kuten — Fri, 23 Oct 2020 13:02:29 +0000

This is our guest post for CNX Software that explains the basic steps to develop a video conferencing app with Gstreamer on TV boxes running Linux.

The COVID-19 pandemic has become a catalyst for new online services. For example, Zoom became so successful that it overtook IBM in terms of capitalization this month. The software engineers at Promwad were inspired by this success and decided to go even further: what about implementing video conferencing on Smart TV and STBs? Then the users of such an application will have an opportunity to communicate not only at work but also to enjoy remote meetings with friends, cheer for a soccer team, watch a movie together, or do sports with a coach.

For some reason, most digital TV operators do not have such a service, although, from an engineering point of view, all these features can be implemented on set-top boxes based on Linux/Android and RDK.

Let’s analyze the architecture of a Zoom-like video conferencing app for Smart TV and implement video stream encoding with GStreamer open-source multimedia framework. We collected information to work with this framework in pieces, but it was worth it.

Challenges and software architecture

When developing the application architecture for set-top boxes, it is essential to consider all potential software and hardware limitations. Engineers who build desktop programs, as a rule, do not face these challenges, while it is a common thing for embedded engineers.

So, what we are going to encounter on set-top boxes:

Limited resources of CPU and devices themselves. In most cases, STB devices use a variety of ARM processors, which implies several limitations and additional tasks, such as the need to use hardware encoding/decoding for a video stream. So, performance is a bottleneck.
Different architectures in STBs by various manufacturers. Some are based on Android; others use RDK or Linux-based distributions with their limitations and nuances. Therefore, it is better to choose the most common and cross-platform solutions in different software modules at the start of the development process. Not to mention the support of a desktop-version. And then move on to some particular cases.
Network limits. Many set-top boxes work both over Ethernet and Wi-Fi. Compression and transmission of video/audio streams is another bottleneck in applications of this kind.
Safety of stream transmission and other data security issues.
Support for cameras and microphones on embedded platforms.

Now we can cluster the architecture itself. Our video conferencing app for set-top boxes will consist of several large components and modules:

Video stream capture
Audio stream capture
Network module
Video/Audio Stream Encoding Module
Video/Audio Stream Decoding Module
Displaying a video conference on a screen
Sound output
Color transformations
Several other secondary components

In a simplified form, the architecture can be depicted as follows:

In this article, we will focus on video stream decoding/encoding and possible implementation with the GStreamer framework, as it is one of the critical points in video conferencing app development.

Audio/video stream encoding and decoding

Benefits of GStreamer for video conferencing

As we have already noted, video streaming is one of our bottlenecks. Suppose you have a camera that outputs footage at 30 frames per second at a small resolution of 640×480. In total, it turns out in RGB24: 640 * 480 * 3 * 30 = 27,648,000 bytes per second, i.e., more than 26 megabytes per second, which won’t work for obvious reasons, notably network transmission bandwidth.

One of the solutions is to implement video encoding using some library. As you can guess from the title of this article, our choice fell on the GStreamer framework. Why this particular library? Here are some of its advantages against other solutions:

Good cross-platform solution with excellent support for Linux and Android.
On RDK, Gstreamer is an encoding/decoding standard included in the default distribution.
It supports a wide range of modules, filters, and codecs. For example, FFmpeg that can be used for the same purposes is one of GStreamer’s modules.
Easy to build a pipeline. It’s easy to create a chain of encoding/decoding, and the pipeline approach enables smooth replacement of codecs, filters, etc. without the need to rewrite the code.
C/C++ API is included.
Support of hardware encoders/decoders, in particular, the OpenMAX API – a significant thing to work with set-top boxes.

Exploring GStreamer and pipelines

Before proceeding to code review, let’s look at what we can do without it. GStreamer includes useful utilities to work with, in particular:

gst-inspect-1.0 will allow you to see a list of available codecs and modules, so you can immediately see what will do with it and select a set of filters and codecs.
gst-launch-1.0 allows you to start any pipeline. GStreamer uses a decoding scheme where a stream passes through different components in series, from source to sink output. You can choose anything as a source: a file, a device, the output (sink) also may be a file, a screen, network outputs, and protocols (like RTP).

A classic example of playing an mp4 file:

gst-launch-1.0 filesrc location=file.mp4 ! qtdemux ! h264parse ! avdec_h264 ! videoconvert ! autovideosink

The input accepts the mp4 file, which goes through the mp4 demuxer — qtdemux, then through the h264 parser, then through the decoder, the converter, and finally, the output.

You can replace autovideosink with filesink with a file parameter and output the decoded stream directly to the file.

Programming an application with GStreamer C/C++ API. Let’s try to decode

Now when we know how to use gst-launch-1.0, we are doing the same thing within our application. The principle remains the same: we are building in a decoding pipeline, but now we are using the GStreamer library and glib-events.

We will consider a live example of H264 decoding.

Initialization of the GStreamer application takes place once with the help of gst_init (NULL, NULL);

If you want to see what’s happening in detail, you can set up a logging level before the initialization.

gst_debug_set_active(TRUE);
gst_debug_set_default_threshold(GST_LEVEL_LOG);

Note: no matter how many pipelines you have in your application, it is enough to initialize gst_init once.

Let’s create a new event-loop where events will be processed:

GMainLoop *loop;
loop = g_main_loop_new (NULL, FALSE);

And now we can start building our pipeline. Let’s name the necessary elements, in particular, the pipeline itself as the GstElement type :

GstElement *pipeline, *source, *demuxer, *parser, *decoder, *conv, *sink;

pipeline = gst_pipeline_new ("video-decoder");
source   = gst_element_factory_make ("filesrc",       "file-source");
demuxer  = gst_element_factory_make ("qtdemux",      "h264-demuxer");
parser   = gst_element_factory_make ("h264parse",      "h264-parser");
decoder  = gst_element_factory_make ("avdec_h264",     "h264-decoder");
conv     = gst_element_factory_make ("videoconvert",  "converter");
sink     = gst_element_factory_make ("appsink", "video-output");

Each element of the pipeline is created via gst_element_factory_make, where the first parameter is the type and the second is its conditional name for GStreamer, on which it will later rely (for example, when issuing errors).

It would also be nice to check that all components are found otherwise gst_element_factory_make returns NULL.

if (!pipeline || !source || !demuxer || !parser || !decoder || !conv || !sink) {
    // one element is not initialized - stop
    return;
}

We are setting the same location parameter via g_object_set:

g_object_set (G_OBJECT (source), "location", argv[1], NULL);

Other parameters in other elements can be set in the same way.

Now we need the GStreamer message handler, let’s create the corresponding bus_call:

GstBus *bus;

guint bus_watch_id;
bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
bus_watch_id = gst_bus_add_watch (bus, bus_call, loop);
gst_object_unref (bus);

gst_object_unref and other similar calls are needed to clear selected objects.

Then we will name the message handler itself:

static gboolean
bus_call (GstBus     *bus,
          GstMessage *msg,
          gpointer    data)
{
  GMainLoop *loop = (GMainLoop *) data;
  switch (GST_MESSAGE_TYPE (msg)) {
    case GST_MESSAGE_EOS:
      LOGI ("End of stream\n");
      g_main_loop_quit (loop);
      break;

    case GST_MESSAGE_ERROR: {
      gchar  *debug;
      GError *error;

      gst_message_parse_error (msg, &error, &debug);
      g_free (debug);
      LOGE ("Error: %s\n", error->message);
      g_error_free (error);
      g_main_loop_quit (loop);
      break;
    }

    default:
      break;
  }
  return TRUE;
}

And now the most important thing: we collect and add all the created elements in a single pipeline, which was built through gst-launch. The order of addition is, of course, important:

gst_bin_add_many (GST_BIN (pipeline), source, demuxer, parser, decoder, conv, sink, NULL); gst_element_link_many (source, demuxer, parser, decoder, conv, sink, NULL);

We should also note that this linking of elements works perfectly for stream outputs, but in the case of playback (autovideosink) requires additional synchronization and dynamic linking of the demuxer and parser:

gst_element_link (source, demuxer);
gst_element_link_many (parser, decoder, conv, sink, NULL);
g_signal_connect (demuxer, "pad-added", G_CALLBACK (on_pad_added), parser);

static void
on_pad_added (GstElement *element,
              GstPad     *pad,
              gpointer    data)
{
  GstPad *sinkpad;
  GstElement *decoder = (GstElement *) data;

  /* We can now link this pad with the sink pad */
  g_print ("Dynamic pad created, linking demuxer/decoder\n");

  sinkpad = gst_element_get_static_pad (decoder, "sink");
  gst_pad_link (pad, sinkpad);
  gst_object_unref (sinkpad);
}

A dynamic connection makes it possible to determine the type and number of threads in contrast to a static one and will work in some cases when it is required.

And finally, let’s turn the conveyor status into a playback: gst_element_set_state (pipeline, GST_STATE_PLAYING);

And let’s run event-loop: g_main_loop_run (loop);

After this procedure, everything needs to be cleaned:

gst_element_set_state (pipeline, GST_STATE_NULL);
gst_object_unref (GST_OBJECT (pipeline));
g_source_remove (bus_watch_id);
g_main_loop_unref (loop);

Choosing encoders and decoders. Fallbacks.

There’s more to tell about useful but barely mentioned things in the documentation: how you can easily organize a fallback decoder or encoder.

The gst_element_factory_find function will help us do this by checking if we have a codec in the elements factory:

if(gst_element_factory_find("omxh264dec"))
  decoder  = gst_element_factory_make ("omxh264dec",     "h264-decoder");
else
  decoder  = gst_element_factory_make ("avdec_h264",     "h264-decoder");

In this example, we have prioritized the selection of an OMX hardware decoder on the RDK platform, and in case of its absence, we will choose a software implementation.

Another extremely useful but even more rarely used feature is to check what we actually initialized in GstElement (which of many codecs): gst_plugin_feature_get_name(gst_element_get_factory(encoder))

You can do it in such a simple way and return the name of the initialized codec.

Video color models

We can’t help but mention color models as well since we are talking about encoding video from cameras. And that’s when YUV goes on stage (much more often than RGB).

Cameras simply love the YUYV color model. But GStreamer likes to work with the usual I420 model much better. If it is not about outputting in the gl-frame, we will also have I420 frames. Get ready to set up the filters you need and perform the transformations.

Some encoders can work with other color models as well, but more often, these are exceptions to the rule.

We should also note that GStreamer has its own module for receiving video streams from your camera, and it can be used to build a pipeline, but we will talk about it some other time.

Let’s deal with buffers and take data on the fly

Input buffer

It’s time to deal with the data flows. Until now, we have simply encoded through filesrc what is in the file and displayed everything in the same filesink or on the screen.

Now we will work with the buffers and the appsrc / appsink inputs and outputs. For some reason, this issue was hardly taken into account in the official documentation.

So how to organize a constant data flow in the created pipelines, or if to be more precise to the output buffer and get an encoded or decoded output buffer? Let’s say we got the image from the camera and we need to encode it. We have already decided that we need a frame in the I420 format. Let’s say we have it, what’s next? How do I pass a picture through the whole pipeline flow?

First, let’s set up the need-data event handler, which will be started when it is necessary to feed data into the pipeline and start feeding the input buffer: g_signal_connect (source, "need-data", G_CALLBACK (encoder_cb_need_data), NULL);

The handler itself has the following form:

encoder_cb_need_data (GstElement *appsrc,
                      guint       unused_size,
                      gpointer    user_data)
{
  GstBuffer *buffer;
  GstFlowReturn ret;
  GstMapInfo map;

  int size;
  uint8_t* image;

  // get image
  buffer = gst_buffer_new_allocate (NULL, size, NULL);
  gst_buffer_map (buffer, &map, GST_MAP_WRITE);
  memcpy((guchar *)map.data, image,  gst_buffer_get_size( buffer ) );
  gst_buffer_unmap(buffer, &map);
  g_signal_emit_by_name (appsrc, "push-buffer", buffer, &ret);
  gst_buffer_unref(buffer);
}

You might say that “image” is the pseudo-code of our image buffer in I420.

Next, we create a buffer of the necessary size through gst_buffer_new_allocate, which will correspond to the size of the image buffer.

With gst_buffer_map we set the buffer to write mode and use memcpy to copy our image to the created buffer.

And finally, we signal GStream that the buffer is ready.

Note: it is essential to use gst_buffer_unmap after writing and to clear the buffer after using gst_buffer_unref. Otherwise, there will be a memory leak. In the low number of available examples, no one was particularly concerned about memory usage, although this is very important.

Now, when we are done with the handler, one more thing to do is to configure caps on the receipt of our expected format.

This is done before installing the need-data signal handler:

g_object_set (G_OBJECT (source),
              "stream-type", 0,
              "format", GST_FORMAT_TIME, NULL);

g_object_set (G_OBJECT (source), "caps",
              gst_caps_new_simple ("video/x-raw",
                                   "format", G_TYPE_STRING, "I420",
                                   "width", G_TYPE_INT, 640,
                                   "height", G_TYPE_INT, 480,
                                   "framerate", GST_TYPE_FRACTION, 30, 1,
                                   NULL),
              NULL);

Like all GstElement parameters, the parameters are set via g_object_set.

In this case, we have defined the stream type and its caps — the data format. We are specifying that the appsrc output will receive the I420 data with 640×480 resolution and 30 frames per second.

Frequency in our case, and in general, does not play any role. While working, we haven’t noticed that GStreamer somehow limits need-data calls by frequency.

Finished, now our frames are fed into the encoder.

Output buffer

Now let’s find out how to get an encoded output stream.

We connect the handler to the sink pad:

GstPad *pad = gst_element_get_static_pad (sink, "sink");
gst_pad_add_probe  (pad, GST_PAD_PROBE_TYPE_BUFFER, encoder_cb_have_data, NULL, NULL);
gst_object_unref (pad);

Similarly, we connected to another sink pad event, GST_PAD_PROBE_TYPE_BUFFER, which would be triggered as the data buffer enters the sink pad.

static GstPadProbeReturn
encoder_cb_have_data (GstPad * pad,
                      GstPadProbeInfo * info,
                      gpointer user_data) {
  GstBuffer *buf = gst_pad_probe_info_get_buffer (info);
  GstMemory *bufMem = gst_buffer_get_memory(buf, 0);
  GstMapInfo bufInfo;

  gst_memory_map(bufMem, &bufInfo, GST_MAP_READ);

  // bufInfo.data, bufInfo.size
  gst_memory_unmap(bufMem, &bufInfo);
  return GST_PAD_PROBE_OK;
}

The callback has a similar structure. Now, we need to reach the buffer memory. First, we get GstBuffer, then a pointer of its memory using gst_buffer_get_memory by index 0 (as a rule, it is the only one involved). Finally, using gst_memory_map, we get the data buffer address bufInfo.data and its size bufInfo.size.

In fact, we have achieved the goal of getting a buffer with encoded data and its size. So, we reviewed the key and most exciting components of our Zoom-like video conferencing application for Smart TV and set-top boxes: architecture, encoding/decoding modules with GStreamer, input/output buffers, and color transformations used.

For digital TV operators, such a software platform can become a new subscriber service. And for us, engineers, this is an interesting new embedded project for implementing different STBs based on RDK, Linux, and Android. As for everybody else, this is an opportunity to spend time together watching movies and sports matches, playing sports, and meeting with loved ones during the COVID-19 quarantine or remote work.

This idea with video conferencing service on Smart TV can be further developed, both in terms of engineering solutions and business scenarios. So feel free to share your thoughts in the comments below.

Null pointers in C++: what you can and can’t do

Ivan Kuten — Thu, 30 Jul 2020 12:22:01 +0000

Some software engineers may have been tired of this topic already, but here we picked 7 examples and tried to explain their behavior using the Standard (the latest draft at the time of writing):

struct A {  
int data_mem;  
void non_static_mem_fn() {}  
static void static_mem_fn() {}  
};

void foo(int) {}

A* p{nullptr};

/*1*/ *p;  
/*2*/ foo((*p, 5));  
/*3*/ A a{*p};  
/*4*/ p->data_mem;  
/*5*/ int b{p->data_mem};  
/*6*/ p->non_static_mem_fn();  
/*7*/ p->static_mem_fn();

One obvious yet important point is that p initialized with a null pointer can’t point to a valid object because it’s “distinguishable from every other value of object pointer” (conv.ptr#1).

Example 1

*p;

It’s an expression statement with *p being a discarded-value expression which needs to be evaluated nevertheless (stmt.expr#1). By definition (expr.unary.op#1), the unary operator * “performs indirection,” and the result is “an lvalue referring to the object or function to which the expression points.” It’s clear what semantics is, but not whether there’s a precondition that an object has to exist. A null pointer is not mentioned there even once.

One could try to conclude from the fact that it performs indirection because basic.stc#4 says that

“indirection through an invalid pointer value . . . have undefined behavior”.

However, that exact paragraph contains the definition of an invalid pointer value and refers to basic.compound#3.4, where a null pointer value and an invalid pointer value are listed as different values of a pointer.

There’s also a note in dcl.ref#5 saying that

“the only way to create such a reference would be to bind it to the “object” obtained by indirection through a null pointer, which causes undefined behavior,”

but it’s not clear which part the last clause is referring to. In case it’s “to bind it,” then binding to a non-existing object is undefined behavior, which goes in line with the normative text of that paragraph.

Since the Standard leaves room for interpretation instead of being clear on this particular topic, let’s turn to the core language issues list, where the Core Working Group elaborates wording of the Standard, among other things. There’s a dedicated issue for our topic, where CWG came to an informal consensus (that’s how “drafting” status is defined) that

“p = 0; *p; is not inherently an error. An lvalue-to-rvalue conversion would give it undefined behavior.”

If the “informal consensus” doesn’t sound good enough, there’s another issue dedicated to example 7, where CWG says it should be allowed for that exact reason.

I’ll take into account this consensus in what follows. If a future Standard prohibits indirection through a null pointer like C does (N2176, 6.5.3.2 and footnote 104), then all examples will be rendered to contain undefined behavior.

Example 2

foo((*p, 5));

In order to call foo(), the parameter needs to be initialized, which leads to the evaluation of the operator comma. Its operands are evaluated from left to right, and except for the rightmost, all of them are discarded-value expressions (expr.comma#1). So this example is well-formed, too.

Example 3

A a{*p};

An implicit copy constructor will be picked to initialize a, and const A& needs to be initialized with a valid object in order to call it, otherwise behavior is undefined (dcl.ref#5). However, there’s no such object in our case.

Example 4

p->data_mem;

Expression of this expression statement will be converted to (*(p)).data_mem per expr.ref#2, which designates “the corresponding member subobject of the object designated by the first expression” (expr.ref#6.2). It’s once again not clear whether there’s a precondition that an object has to exist. Seeing “to refer” and “to designate” being used interchangeably in basic.lookup.qual#1 make it similar to example 1 all the more. I’d say that this example is well-formed because of that, but some compilers disagree. See “Checking with constant expressions” section at the end of this article for more details.

Example 5

int b{p->data_mem};

Continuing the previous example, we’ll try to initialize int with the result of expression instead of discarding it. It needs to be converted to prvalue, because expressions of this category initialize objects (basic.lval#1.2). Since the target type is int, the result of the expression will be accessed (conv.lval#3.4), which leads to undefined behavior in our case, because none of the conditions in basic.lval#11 are met.

Example 6

p->non_static_mem_fn();

class.mfct.non-static#1 reads that

“a non-static member function may be called for an object of its class type, or for an object of a class derived from its class type,”

where “may be” means permission, and not a possibility (ISO Directives Part 2). So behavior is undefined since there’s no object.

Example 7

p->static_mem_fn();

As we mentioned in description to example 1, CWG says that this example is a valid code. The only thing to add is that indirection through expression to the left of -> is performed even when its result is not required (foontnote 59).

Checking with constant expressions

Since constant expression can’t rely on undefined behavior (expr.const#5), we can ask compilers’ opinion on our examples. Even though their diagnostics are not ideal, they are at least sometimes right. We edited our examples a bit to fit them into constant expression evaluation, fed them to three popular compilers, and commented out examples they had considered bad because diagnostic messages of GCC and MSVC leave a lot to be desired on those particular examples. The tests themselves can be found on godbolt, and the summary of our results is presented in the table below.

#	Snippet	Expectation	GCC 10.1	Clang 10	MSVC 19.24
1	*p;	+	+	+	+
2	foo((*p, 5));	+	+	+	+
3	A a{*p};
4	p->data_mem;	+	+
5	int b{p->data_mem};
6	p->non_static_mem_fn();		+		+
7	p->static_mem_fn();	+	+	+	+

The results make us a bit doubtful about our conclusion on example 6, and even more on example 4. But it’s also interesting to see all of us share the same opinion about the key example 1.

Thanks for staying with us to follow the adventures of a null pointer in C++! :-) Usually, we share fragments of code taken from our current firmware development projects, but this time we were interested in purely "philosophical" questions, so the examples were synthesized.

If you share our love of contradictions in C++, feel free to share your code and comments.