TVWish Suggestions System

People either love or hate the Tivo "suggestions" system, which has the box regularly recording -- if it has nothing else to do -- shows that it hopes match your tastes. Tivo does this by looking at your recording history, and letting you press Thumbs-Up and Thumbs-Down on shows to cast your vote on how much you like them. The suggestions are shown at the end of the list of programs, and never take disk space or a tuner away from real recordings.

Some people wonder why they want the box recording things they didn't ask it to record, or think the programs are somehow wasting disk space or resources. Some turn them off because they clutter the screen or turn on the recording light at times it wasn't asked to be on.

However, it turns out the suggestions, even if they don't do a very good job, provide a very interesting feature. They are a replacement for surfing. Instead of just browsing whatever is randomly on at a given moment, you can browse a different set of somewhat less random choices gathered recently. They don't take away your ability to surf live TV, but they expand it, and with all the wonderful benefits one only gets with recorded shows, such as FF.

With MythTV

MythTV isn't quite as ready for suggestions as Tivo, but TVWish has a first-draft at doing this. Put in some tastes, or a request to learn from past recordings, and a Suggest-Go wish, and a set of suggestions will be chosen from the upcoming day and queued for recording.

TVWish Suggestions are set with a very low recording priority (-90 or below.) This means they never interfere with actual scheduled recordings. They are also set with a very high (100 or above) "Autoexpire" value so that they are always autoexpired first, before any actual requested recordings. In this way, they don't interfere with regular recordings. They are not displayed specially, and they are listed with the other recordings instead of on their own. This should change in the future. They all go into the TVW-Suggest "Recording Group" so you can see them as a group.

When a Suggestion is recording, it does use up a tuner. If you need the tuner for live TV you must stop recording on and delete the suggestion. This is another thing for the tudo list. Suggestions should be otherwise hard to notice until you browse them.

Note: Suggestions require "programid" fields in your listing database. Users in North America have programids. In other nations, there may not be programids present, especially in older versions of MythTV, depending on how your listing data arrives.

Quick Basic Install

Just add two lines to your master TVWish file:

Suggest-Learn:
Suggest-Go:

Requesting them

Simply add wishes that start with "Suggest-" to a TVWish file. Critics can also trigger suggestions, but can't do the Suggest-Go. Normally you will have one or more wishes to specify preferences, and then finally a single "Suggest-Go" to trigger the scheduling of suggestions. (Indeed, a second Suggest-Go will do nothing.)

Suggestion wishes all get a "weight" on them, which affects how likely programs matching the wish will be picked. The defaut weight is 10. For every suggestion, you can add "weight=nn", or just put a number (floating point if desired) at the end of the line.

Suggest-Learn:: Tells the system to learn about your choices by scanning previously recorded shows. Right now it mainly learns the genres and titles of the shows and gives them all low weights.
Suggest-Go: [hours=num] [commflag=0|1] [random=num]: Queues up suggestions based on all specified preferences. You should have one, and only one call to this command, typically at the end of the file.
Suggest-Genre: Genre: Suggests programs with the given genre.
Suggest-Title: Title: Suggests shows with that title. This is the name of a series, not the name of an episode. Use regular TVWish for specific episodes. Often a negative weight might be used to stop a series from being suggested.
Suggest-word: keyword: Suggests shows whose title, subtitle or description contains the keyword.
Suggest-person: name: Suggests shows where the named person worked on the show (actor, director, production)
Suggest-pilot:: Suggests shows that appear to be pilot episodes of series. Only a rough test is used, not as good as the TVWish "First" command.
Suggest-movie:: Suggests movies of better than 2 stars. The weight will be based on the number of stars, so a 4-star movie is suggested almost twice as much as a two-star.

Example

Suggest-Learn:
Suggest-Person: Mark Burnett   weight=4
Suggest-Genre:  Science fiction  weight=15
Suggest-Word:   Burning Man
Suggest-Pilot:           
Suggest-Movie:            20
Suggest-Go: hours=4

The above file will request 4 hours per day of suggestions, based on prior recordings but with special preferences of all sorts. The high weight of movies (4 star movies will earn 22 points) will make them the most common choices.

Detailed Rules

Suggest-Go

This is the most complex command which actually queues up the suggestions. It processes one day of programming, starting at the end of the last suggested period (or now, if that's in the past) and going forward for one day. If the start of this period is more than 18 hours in the future, no suggestions will be processed. This allows you to run TVWish several times a day, but only do the suggestion processing once a day. Only the immediate future is done to avoid cluttering your recording list with far future suggestions.

The suggestion process scans over these shows and adds scores to them based on the weights of various factors. In addition, 30 programs resulting from the first scan have a random adjustment made to their score to add some deliberate randomness to the system.

By default, approximately 3 hours of suggestions will be scheduled. You can change this amount with the "hours=nn" tag. However, 3 times the number of planned hours of the best scoring programs will be chosen, and from those, the number of planned hours will be taken at random.

No series will be suggested more than twice in one day, and shows or series that have been suggested in the past 4 days will have their score seriously reduced to avoid repetition.

By default, suggested shows are not set for commercial flagging, even if you turn that on by default. This is to avoid having the suggestions create a large CPU burden. When you watch them, you can turn on live flagging. If you would prefer to have suggestions flagged, add "commflag=1" to your Suggest-Go command.

As noted, 30 shows have their score adjusted by something in the range +/- 10 points at random. You can change the range of the randomness with "random=nn", or you can turn of this randomness with "random=0". You can't turn off the random selection of top ranked shows from a pool 3 times larger than the planned number of hours.

While Critics can change the weights of suggestions, critics can not do the Suggest-Go command. If you wish critics to alter your suggestions, you should include their URLs and make sure you include a Suggest-Go in the file that inserts the critic. (Bug: Critic suggestions won't work right now.)

Suggest-Learn:

This gathers up all past recorded, currently stored and scheduled shows and adds weights based on their genres and titles. Later we hope to be able to add weights based on people. The weight added is typically 4 to 10 points for a genre and 2 points for a title. Of course, you may want the reverse when it comes to titles, not wanting to see suggested shows you have already watched or are currently recording. Right now you must manually reduce the weight for those titles.

You can place a weight argument to change it from the default of 10. This will control just how much effect the learned choices have on your suggestions. Suggest-Learn: 20 will double the effect, Suggest-Learn: 5 will halve it.

Suggest-Genre: genre

Sets a preference for genres (or a negative weight for an aversion.) All programs can have several genres. The genres are ordered, so there is a "main" genre and secondary ones at various levels. If a program has your requested genre as the main genre, you will get 100% of the weight. At the second level, 66% is given, then 50% and so on descending.

You must match genre names exactly. Currently you can get a list of genere names in MythTV in its recording section. If you use a nonexistent genere, you will get a warning.

Suggest-Title: title

Sets a preference for a particular show. This will be the name of a series, not an episode, or the name of a special or movie. As noted, if the learn mode is over-suggesting a particular show, consider reducing the score as follows:

Suggest-Title: The Apprentice weight=-2

Suggest-Word: Keyword

Searches the titles, subtitles and descriptions for the keyword or phrase. Case is not important. This is the slowest suggestion to process, however. You can also have MythTV or TVWish explicitly record based on keywords. The suggestion mode of course makes things more random. You can also apply a high weight but at most a show only has a 33% chance of making it.

Suggest-Person: Name

If you want to follow a particular actor or director or other moviemaker, you can add a weight to shows they were involved in. You will get a warning if the name you provide has never shown up in listing data in all your time using MythTV.

Suggest-Channel: Callsign

Lets you specify a weight to apply to programs on a specific channel. You use the short callsign for a channel, like DISNP (Disney Channel Pacific) that you often see in listings. This may be used both to love programs from a particular channel, or to exclude them with a negative weight. Since channel searches will match a lot of programs, use this sparingly.

Suggest-Pilot:

Searches for pilot episodes (based on a basic episode-ID test) and gives them an automatic score boost. Nice if you would like to have a random selection of first episodes. The Pilot search is not as fancy as the First wish of TVWish. It may be improved over time. If you are really looking for a pilot of a particular show, it is recommended you explicitly wish for it.

Right now it needs the programids from DataDirect for most detections, so don't expect a lot outside North America.

Suggest-Movie:

Provides a preference for movies, especially 4-star movies. Two 1/2 star movies will get about half the weight added to them, 4 stars a little more than the weight specified. Movies 2 stars or less are never suggested this way.

Tips and Traps

If you want to keep a suggestion, go to it in your MythTV program menu, and set it to "not autoexpire." (Unfortunately, if you set it back to autoexpire again the TVWish expire hack will put it in the suggestions autoexpire queue.)

As noted, suggestions are meant to be random. You can't be sure to get something with a suggestion. If you want it explicitly, add an explicit wish for it.

Right now the system will schedule recordings even if they are on at the same time as regularly scheduled recordings. Thus, they will lose the record priority battle. In such cases, you will not get the full number of hours requested. Indeed, the system currently will schedule two suggestions at the same time. One of them will win, following MythTV rules. (Todo: Make it the higher scoring one.)

As noted above, the suggestions, while they don't interfere with recordings and expire first, otherwise appear as normal recordings. The plan is to make them a bit different, perhaps grouped at the bottom of lists, or in a different colour. They do have their own recording group. It is recommended you go into MythTV settings and ask that you see Recording Groups as part of your menus of current recordings.

As noted, suggestions, when recording, tie up a tuner that you might watch live TV with. You must go and cancel the recording of the suggestion, and then optionally delete it. You don't have to delete it due to the high autoexpire score.

Note: Due to a bug in MythTV, high autoexpire scores scheduled for recodings are not passed through after they are recorded. TVWish corrects this for all recordings in a recording group starting with "Suggest." A patch for MythTV has been produced to fix this bug.

The use of high autoexpire scores means your disk will always appear full, even though suggestions will be automatically removed for any real recording. Eventually the display screen may be adapated to break down disk space usage based on suggestions and regular recordings. Unix disk space tools will of course show the partitions mostly full.

To Do

Weights assigned to HD programs, specials, non-repeats and more.

Direct SQL queries for suggestion scans.

Detection of conflict with existing recordings and other suggestions

Mods to MythTV to treat programs that have either low RecPriority or high AutoExpire differently -- different display, different groupings, not counted in disk space summary etc.

Mods to MythTV to remember secondary genres, credits of old recordings.

Mods to MythTV to have record/recorded/oldrecorded be able to tell if a recording was a suggestion or not. (Recpriority/autoexpire rememberance is enough.)

MythWeb interface to all of TVWish including suggestions.

Thumbs up/Thumbs down interface in MythTV.