Background: The human-device interaction has seen unprecedented changes in the past few years.  Today consumers are demanding in several ways and the patience to wait for gadgets to respond to our commands, instructions and touch is reducing.

Smartphones have set a different level of expectations for human-device interaction.  The same has got extended to Tablets as well as Laptops.  Over the past 2 years, the Tablet industry has been working on extending the smartphone experience on this form factor especially the ones powered by Android. 

In laptops, one of the key attractions for consumers preferring MacBooks has been its superfast responsiveness, especially the boot time, which is almost instant as soon as a user turns a MacBook on.

The only smart device which is still going by legacy standards and hasn’t been yet able to catch up with the user expectations is Smart TV.  Though some leading OEMs have done their own interventions to deliver very responsive interactions, yet the large base using Android and Google OS powered Smart TVs is deprived of this experience.

Lumio, a homegrown brand by Circuit House Technologies has taken this mission and claims to have addressed this major gap in the industry.  They approached us to validate their claim by benchmarking the responsiveness / speed of their Smart TVs against the competition Smart TVs powered by Android and Google OS.

TechMark Benchmarking: We designed a framework and process for the same under our TechMark benchmarking offering.  To validate Lumio’s claim we identified 4 key human-device interactions on a Smart TV that define the speed / responsiveness experience.  These were

– Booting of Smart TV.

– Using OTT applications.

– Installing apps.

– Scrolling through the TV menus.

To benchmark Lumio Smart TVs performance with the competition Smart TVs, the following process was defined.

• 4 tests were defined for measuring the performance on the above interactions.
  • Boot time was defined as the time from triggering ON to the screen when TV is ready for interactions.
  • For OTT applications, we chose Netflix, one of the heavies and popular OTT platforms for entertainment. The responsiveness of OTT apps was measured from clicking ‘open’ to the time when the profile menu appeared on the screen for users to access their favourite content.
  • In case of app installation, we chose Plex as it is one of the heaviest applications to install on a Smart TV. In this test, the time taken to install the application and be ready for opening was measured.
  • For scrolling test, we chose the settings menu and measured the time taken to scroll from the topmost item in the menu to the last item in the list.
• A list of competition devices was prepared. To ensure comprehensive coverage of competition Smart TVs, we chose top 100 best-selling Smart TVs using Android / Google OS priced under Rs 1 Lakh. These 100 Smart TVs contribute to over 85% of the total Smart TV sales powered by Android / Google OS. Some of the notable brands covered in this benchmarking included Sony, Xiaomi, Redmi, TCL, Acer, Panasonic, Hisense, Toshiba, Vu, VW, Croma, Motorola, Haier, Intex, Kodak and Foxsky. A total of 25 such brands were captured in the process. To get the details of all the brands and models, please request here.
• A test guide was prepared for the field technicians with instructions on how to conduct these tests at user premises that included a mix of hotels, retail outlets and residential consumers.
• The technicians were also demonstrated how to carry these tests.
• The criteria for selecting these Smart TVs included that they should be from the top 100 models identified, and the purchase should not be older than 3 months.
• Our field technicians visited various premises and after seeking permission as per the market research guidelines, conducted these tests on the Smart TVs of various users. No personal identifier information was captured during this exercise.
• Lumio TVs were tested by us for the same tests. The tests were faciliated by Lumio team.
• The results were collated and analysed further.
• Lumio TV test results were compared with that of industry. The results of competition Smart TVs were averaged (mode) to measure the time taken for similar activities by most of the competition Smart TVs.
• Accordingly, the performance with regards to speed / responsiveness of Lumio Smart TVs was benchmarked against the industry (competition) Smart TVs.

Results

In all the 4 major tests performed to measure the speed / responsiveness of Lumio TVs against competition, the time taken by Lumio Smart TVs was significantly less than that of the industry benchmark.  Overall, the existing industry Smart TVs took anywhere between 2 – 4 times more time to perform same activities than Lumio Smart TVs.

Below are some of the key test findings.

• Lumio Smart TVs took 33 seconds to boot up and be available for users to interact with.  Against this, most of the competition Smart TVs took 67 seconds for them to reach the same state (screen).  Some Smart TVs even took 74 seconds for this operation, while the Smart TV that came close to Lumio Smart TV still took 44 seconds for rebooting.
• For the OTT test, it took just 13 seconds on Lumio Smart TV to reach the Netflix profile screen, where the users could access their content as per their preferences and choice.  The competition Smart TVs consumed 40 seconds for the same operation.  The slowest Smart TV here took 45 seconds to reach the profile screen while the fastest among competition Smart TVs took 35 seconds for reaching the same state.
• In case of app installation, it took only 33 seconds for Lumio Smart TVs to install the Plex app, while the competition Smart TVs took average 131 seconds.  Within competition Smart TVs it took 146 seconds for one of the Smart TVs to install the same app.  Among competition Smart TVs the fastest any TV could install the same app took 120 seconds.
• To measure the smooth flow of the Smart TV, it took Lumio Smart TVs only 6 seconds to scroll from top to last item on the menu in settings. The competition Smart TVs took average 17 seconds.  The slowest Smart TV from competition brands took 25 seconds for the same while the fastest among them took 11 seconds for scrolling from top to bottom of the settings menu.

Additionally, during TechMark benchmarking tests, it was also noticed that the frame drops were less than 5 in case of Lumio Smart TVs playing 4K video for over a minute, as against the competition Smart TVs that showed average 55-60 drops for the same duration.

Takeaway

The TechMark benchmarking for Lumio Smart TVs compared with 100 top selling Smart TVs using Google or Android OS, which contribute over 85% of sales volume in India show that the speed / responsiveness of Lumio Smart TVs is significantly faster than the available options from the competition.  These tests not only validate the claims made by Lumio but also make them India’s fastest Smart TV brand.

It is expected that this Smart TV will address one of the smooth TV viewing experiences of users who find slow responsiveness of their Smart TVs as one of the top 3 issues impairing their overall satisfaction with the Smart TVs.