Michelle and I are thinking of moving. When we get to our new place, I want to start to lay out the ground work for home automation, and get as much set up as possible. The end goal seems simple enough. I want automatic context based lighting in the living room. There’s just one little problem: all our “smart” TV technology is actually really dumb.
There’s a lot more to be said for my home automation dreams. Certainly enough to fill many other blog posts. For now, let’s turn our attention to the living room. Before I explain what I meant by “context based lighting”, let’s go over the setup.
The Lavado Living Room
- TV - Samsung LN46B540
- 46” LCD TV, from 2009. Has 4 HDMI ports.
- Audio - Logitech Z-5500
- 5.1 System, Optical In, Coax In, 6 Channel Direct. THX Certified, hardware Dolby Digital, DTS, DTS 96⁄24 support.
- HDMI Switch - Monoprice HDX-401TA
- 4 HDMI In, 1 HDMI Out. Splits out Optical, Coax, and Analog audio.
- Media Player - Apple TV, 4th Generation w/Siri Remote
- Game Console - Microsoft Xbox One, Day One Edition
- Limited Edition, with Controller and Kinect.
There were other devices connected before (e.g. Xbox 360, PS3, others), but those have been unplugged and placed in storage. The current setup is what I would call the “minimum viable home theatre” for us. It contains all the things we will use on a regular basis.
The Apple TV and Xbox both connect to the HDMI Switch. The HDMI Switch separates the audio stream, and passes it through all its outputs simultaneously, but only the TOSLINK/Optical Audio port is connected. The optical signal goes to my Logitech Z-5500s, which then performs any necessary hardware decoding before playback. The audio split is crucial - there is only one optical in on the speakers’ control pod, and there are two sources that need to bring audio through. The switch wouldn’t be required if I perhaps had a newer TV - this 2009 Samsung model is unable to passthrough 5.1 audio through its optical out, only stereo1. With this method, I ensure I get the audio directly from the source. It also has the benefit of allowing me to AirPlay music without having the TV on, as the audio gets split directly at that stage.
The HDMI Switch’s single output port goes to HDMI 1 on the TV (the primary port on the back). Whenever we turn the TV on, it’s pretty much locked just to this input, with the internal speakers disabled as well.
The Harmony of Activities
Everything centers around our remote control, the Logitech Harmony 700.
I picked it up for a song many years ago, during a Best Buy Boxing Day sale. Its been with me since at least 2010. The remote has been dutifully reprogrammed over the years, and it’s included low self-discharge nickel–metal hydride battery (LSD NiMH, or “Eneloop”) batteries are still running strong. They get recharged on average about once every two months, with our current usage.
It’s programmed with three main activities:
- Watch A Movie/Watch TV
Turns on TV, HDMI Switch, turns on Apple TV via IR, turns on speakers, sets correct inputs.
- Listen To Music/Listen To Digital Music
Turns on HDMI Switch, turns on Apple TV via IR, turns on speakers, sets correct inputs.
- Play Xbox One
Turns on TV, HDMI Switch, turns on Xbox One via IR, turns on speakers, sets correct inputs.
It’s simple, reliable, and repeatable. The hallmark of the Logitech Harmony remotes is their “Harmony Technology”, or essentially, internal state tracking. It keeps track of what devices are turned on and what state/input they are in during any activity. This allows for automatic task slimming as well: if I go from the Apple TV to the Xbox, it knows it just has to turn on the Xbox and switch the HDMI input, not reset everything.
The Harmony remote has introduced stability into our media consumption as well. It’s one interface that my wife and I know very well, down to the point where we can press the buttons without ever looking at it directly. We know what it will do, and if things don’t turn on in the right order, or to the correct input (rare), then everyone knows how to use the help function to get things back on track.
The Dream of Lighting
I like to write out my goals as a mini story of sorts. Here’s the version I currently tell, which has evolved over the years.
It’s a typical evening at the Lavado household. Dinner was delicious, and now it’s time to gather on the couch and catch up on news, current events, maybe watch a YouTube video, a TV Show, or maybe even a movie on Netflix. You reach for the remote, and turn the TV on. If they weren’t already on, the lights come on in the area, to a reasonably bright level. You navigate to an app of your choice, find the content you want to watch, and hit play. The lights dim, and the show starts.
The phone rings. Not a problem. You hit pause, and the lights fade up so you can pick up the call, and head to the kitchen to grab a drink while you’re at it. When you return, just press play, and the lights fade down again automatically.
When the show is over, you hit the menu button to go back to the main screen, and the lights go up, now that you’re no longer watching anything. You press the “Off” button on the remote, and everything goes to sleep.
This is the (simplified) version of what I’d like to achieve, but it’s the one that illustrates the problem at hand: How do I know the TV is on, and how can I tell if it is being used to watch video?
There are a few approaches here, which I’ll try to unpack in as much detail as possible.
With Current Equipment
These scenarios will be based on what we currently have on hand. I have a very, very extensive section about future equipment choices drafted in my mind.
Activity Start/End
The first stage is knowing if the TV/Systems are on or not. With our current equipment as the limitation, this will have to consist of “passive” detection. I’ll need to have something that can receive IR at the TV area, that will listen for a set of preconfigured codes. Let’s call them “Activity 1 On/Off, Activity 2 On/Off, Activity 3 On/Off” in English, but they would be some currently unused IR commands in real life.
I can program these codes into the Harmony, and have them fire off during the start and end of any activity. The “receiver” or “listener” could be as simple as an ESP8266 with an IR Receiver attached. This could dispatch the received signal over the network to whatever is currently controlling the home.
Realistically, it will probably be a Raspberry Pi (RPi) with an IR receiver like FLIRC, listening with LIRC. The Raspberry Pi will be in the TV area because I eventually plan to set up some sort of ambilight (again, another blog post). Once the signals are received by the Raspberry Pi, it can be configured to fire off a trigger to any service I like, whether it’s a self-hosted solution, or a cloud solution (like most home hubs).
Media State
Okay, so we know we’re in an activity. Let’s presume this is watching a TV Show using the Apple TV, so the lights are currently in a regular ON state, not dimmed in any particular fashion. The next step is for us to choose a TV Show, and start playback, thus dimming the lights further. The fourth generation Apple TV still includes an IR sensor, so it can be controlled by the Harmony. We’ll navigate over to the app, open it, open the TV folder, and locate the episode to watch. Here we have two choices to start playback: we can either press the “Select/OK” button, or we can press the “Play” button (the Harmony remote in question has separate Play and Pause buttons).
Upon pressing the Play button, two signals can be fired very quickly: the play command, and then a similar “Activity 1 Playing Media” command, to be picked up by the RPi. This would start the appropriate dimming scene, and turn the lights down low for our “critical viewing”. Similarly, when the Pause button is pushed, a separate command can be dispatched that turns the lights back up to normal. I haven’t decided if this will happen instantly, or if it will be a slow, gradual brightness increase.
Here we reach Problem No. 1. The Siri Remote.
Problem No. 1 - The Siri Remote Is Cool
When we used our second generation Apple TV (jailbroken, naturally), it used an IR remote. The regular Apple TV IR remote is nice, but it certainly didn’t offer very much by way of unique functionality. This allowed me to “sell” the Harmony remote. One remote would take care of all the activities, set everything correctly, and it could control the Apple TV! It was a win for everyone involved. Now with the addition of the Siri Remote in the mix… we actually like to use it more.
Right now, the Harmony remote has been delegated to the role of “device starter/stopper”. It’s used to fire off the activity functions, turn stuff on and to the correct inputs, then… it just sits there. We love the touchpad on the Siri Remote, and it makes using the UI pretty fun. I love swishing past selections, and so does my wife. We’ve gotten the gestures down to a science, skipping forward or backward, scrubbing through the timeline, and I even fire off the occasional Siri search. To compact things even further, I programmed the volume control for my Logitech speakers in to the Apple TV, so the remote’s volume control works properly. It’s become a one-stop shop for our media consumption device of choice.
To perform all of this magic, the Siri Remote communicates primarily through Bluetooth to the Apple TV. This means that there is no way for me to intercept a button press. I can’t passively detect if the Play/Pause button has been pushed at all.
We could switch to using the Harmony, but it might be a tough sell. I just turned to Michelle and asked which remote she would prefer using. First response: Apple TV Remote (Siri Remote). The reasons she likes it? “I find it’s easier to use… I can fast forward to a particular time… I don’t have to look down to press the right button.”2
This means we might be stuck. Time for the solutions.
Solution No. 1 - Dumb Dimming
In the “Dumb Dimming” scenario, we only adjust the light levels at the start and end of an activity. When we start watching something on TV, whether it’s a show or a movie, we don’t get up too often. We have usually brought our drinks and snacks over to the living room just before we turn on the TV. In this case, I would have the lights gradually fade down, maybe over five minutes. I would keep it bright enough that we could still get up and make our way around if needed. When we turn off the TV, the lights would then go up to full brightness very quickly.
Solution No. 2 - Button Based Dimming
In this scenario, which can complement the first solution, there is an added control near the sofa, or built in to the Harmony’s buttons, that can always set a few different scenes, and adjust brightness. This could be something like the Logitech Pop, the FLIC, or something else similar. The idea would be that one button press brings us to the “dim, but not too dim” scene, another button press brings us to “very dim, critical viewing”, and finally, a full off. It would be really cool if something like the Lutron Connected Bulb Remote had more configurability for the top “On” button, to allow such scenes to be constructed. Then it could be set to “On, Dim 1, Dim 2”, reserving the bottom button for “full off”, and still allowing discrete buttons for manual adjustments. If I have an IR listener in an RPi, I can map these all to commands on the Harmony remote, but sometimes it’s nice to just have a wall switch for something.
Solution No. 3 - HomeKit, tvOS Edition
Another extension to the above solutions. I initially set up HomeBridge so I could use Siri to control my Lockitron. Along the way, I added my Nest Thermostat and Wink Hub to the mix. HomeKit can have scenes configured into it, so I can set up matching scenes just like above: Dim 1, Dim 2, Off, etc. With tvOS 10, HomeKit will be coming to Siri on Apple TV. This means that the Siri Remote (which we’re already using), will allow me to activate the corresponding scene on demand.
Imagine: “Siri, show me Star Trek: The Next Generation, and set the TV Scene.”
Closing Thoughts (For Now)
I’ve been thinking about this too much lately, so I’m gonna stop the post here and let it take a rest. There’s a lot that’s been temporarily cut: considerations of what new technology would make this easier, what other roadblocks stand in my way, and why I haven’t made any changes to make this easier before. I’ll be back with links to those posts in a few days.
Do you have any thoughts? Share them with me on Twitter, or in the comments below.
- Okay, it actually can pass through 5.1 audio, but only from the internal ATSC tuner. Anything else gets down-converted to stereo for licensing reasons. I’m told newer TVs don’t have this issue and just repeat the signal through. [return]
- Anthony: “Can I quote you on that for my blog post?” Michelle: “Yeah, why not?” [return]