Jump to: navigation, search
diagram of how mythicalLibrarian works
actual picture from XBMC with recordings from MythTV
actual picture from XBMC with recordings from MythTV

The Missing Symlink

mythicalLibrarian will convert your MythTV recordings into a Video library, free of commercials, automatically after recording. You can see an example to the right.

About The Program

mythicalLibrarian is a versitile tool designed to assign season and episode numbers to television shows based on show title and episode title so they are easily recognized by XBMC or Boxee. mythicalLibrarian interfaces with TheTvDb to gain information about the target file. mythicalLibrarian also has the ability to interface with MythTV's local database, XBMC's command interface, and Ubuntu GNOME desktop to provide an all-around recording recognition/managing engine which will run in the background of your back-end computer to update your library and provide status updates upon each completed job.

mythicalLibrarian pulls information supplied by your MythTV program guide for episodes, and makes a fuzzy logic match out of the supplied data. If fuzzy logic is not enough, it will match original air date. For Movies, mythicalLibrarian will rename your MythTV recordings based upon information obtained from the program guide database on your computer. If you allow commercial data to be generated for your shows, mythicalLibrarian will also copy this data with the recordings. mythicalLibrarian allows MythTV to remain in control of the files, so there is no reason not to process your recordings for boxee and XBMC.

MythicalLibrarian was designed to run under Ubuntu Karmic Koala OS through BASH 4.0

Click this link to see an example database

How it works

mythicalLibrarian will generate several files of it's own while running. It must have it's own working directory. To the right, you can see an example of the layout of /mythicalLibrarian, the files which it creates and their purposes. Here is the basic flow:

  • You put in- 'mythicalLibrarian "file/to/process.ext" "episode or movie title" "episode subtitle"
  • Episodes- mythicalLibrarian references TheTvDb to gain SeriesID. Get current time from TheTvDb If the database is out-of-date or non existant, it will download the full series and parse it. Then, using fuzzy logic, it makes a match of the series name and gains season and episode information. If this is not possible, it will attempt a match of original airdate from your MythTV program guide database.

mythicalLibrarian will rename your recognizable episodes to: "/Show Name/Show Name.SxxExx (episode subtitle).ext"

  • Movies- mythicalLibrarian references your MythTV program guide database to verify it is a movie and then obtains the year for the movie.

mythicalLibrarian will rename your movies to: "/Movie Name/movie title (Year).ext"

  • Showings or no TVDB info- mythicalLibrarian will skip all processing and rename the files to "/Show Name/Show Name.S0E0 (episode subtitle).ext"


While it is possible to manually configure the script, this may lead to configuration issues. mythicalLibrarian utilizes an advanced easy installation method which allows you to choose between using the latest stable version, or the current version located on your computer. As of right now, the SVN is updated daily, so take advantage of this easy deployment method to increase your features and functions. All users are encouraged to use the SVN version as the primary and in the event of a diagnostic failure, revert to the last stable version. Updating mythicalLibrarian to the newest version is as simple as typing mythicalLibrarian --update and then hitting enter a bunch of times.

Additional Information

  • If you would like to see additional features and functions, additional back-end support such as tvheadend (and are willing to collect data), or additional guide data type support, please post in this thread


1 Installation Setup and Use of mythicalLibrarian

This section covers installation, setup, and use of mythicalLibrarian. After completing this section, mythicalLibrarian can run autonomously without user intervention. See the Librarian Interfaces for information on notifications and automation tasks.

1.1 Installation

This section covers installation of mythicalLibrarian.

1.1.1 Automatic Installation on linux

This section covers automatic installation and upgrade. Automatic upgrade will allow you to choose between downloading the newest stable, the newest SVN or continuing with the last downloaded version. I encourage SVN usage as it will assist with development.

  • type the following:
sudo apt-get install curl
mkdir ~/.mythicalLibrarian && mkdir ~/.mythicalLibrarian/mythicalSetup && cd ~/.mythicalLibrarian/mythicalSetup
sudo chmod +x ./ && sudo ./
  • mythicalSetup's first prompt will ask you to update the working copy of mythicalLibrarian or to continue with setup. If you download a new version, it will bring you back to the same prompt at which point you will select continue. This gives you the option to upgrade, or just configure the current version.
  • mythicalSetup will run you through a series of prompts and perform the setup of mythicalLibrarian and all of it's features
1.1.2 Installation on Mac OS X

This section covers installation on Mac OS X. Mac OSX Dependencies

This section will cover compiling binaries needed for fuzzy logic capabilities and a terminal-based menu-driven interface

Mac OSX does not have the packages "tre" or "dialog" by default. These packages are required to run mythicalLibrarian and are available through macports. Installation of these packages is required for first time use.

  • Install Xcode Tools

note: You can get Xcode from your OSX installation DVD in "Optional Installs/Xcode Tools/Xcode Tools.pkg" or you can log onto the Mac Dev center at and download it for free.

  • Install X11

note: you can get X11 from your OSX installation DVD in "Optional Installs/Optional Installs.pkg" or you can build it from online sources

  • Obtain and install MacPorts

note: you can get MacPorts here: or navigate to their website here:

  • Open Terminal from Finder:Applications/Utilities/Terminal
  • Type the following
sudo port install tre
sudo port install dialog Automatic Installation on OS X

This section will cover installation of mythicalSetup, which will provide an easy interface to install mythicalLibrarian

  • Type the following into the Terminal
mkdir ~/.mythicalLibrarian && mkdir ~/.mythicalLibrarian/mythicalSetup && cd~/.mythicalLibrarian/mythicalSetup
sudo chmod +x ./ && sudo ./
  • mythicalSetup's first prompt will ask you to update the working copy of mythicalLibrarian or to continue with setup. If you download a new version, it will bring you back to the same prompt at which point you will select continue. This gives you the option to upgrade, or just configure the current version.
  • mythicalSetup will run you through a series of prompts and perform the setup of mythicalLibrarian and all of it's features
1.1.3 Easy Upgrades

This section covers upgrading of mythicalLibrarian.

  • type the following
sudo mythicalLibrarian --update
  • mythicalLibrarian will download the newest version of mythicalSetup and perform the upgrade. mythicalLibrarian will remember your settings from the last time mythicalSetup was run.
1.1.4 Manual Installation for Linux

This section covers manual installation on Linux. Automatic installation is recommended: see below.

  • Install dependencies
sudo apt-get install curl agrep libnotify-bin

dependencies breakdown:

  1. curl downloads web pages as text.
  2. agrep or tre provides fuzzy logic.
  3. libnotify-bin provides GNOME desktop notifications
  • Download the current version of mythcalLibrarian from this download link
  • Place mythicalLibrarianin /usr/local/bin
  • mkdir /home/mythtv/.mythicalLibrarian
  • Make mythicalLibrarian executable by typing the following:
sudo chmod +x /usr/local/bin/mythicalLibrarian
Click this link to see the big picture

This section covers basic setup of mythicalLibrarian. To make mythicalLibrarian work, you will need to set the following settings located in text format within can see a picture of the settings to the right.

  • MoveDir - this is your primary episode move dir.
  • AlternateMoveDir- this is the folder where episodes will be moved if movedir cannot be used
  • FailSafeDir- folder to place links if files cannot be moved
  • PrimaryMovieDir-Folder to place movies
  • AlternateMovieDir- this is where movies will be placed if PrimaryMovieDir fails
  • Database username and password should be mythtv/mythtv. When installing MythTV a root user and password was assigned by the user.

Leave the rest as default, or enable and disable settings as desired.

Extra settings documentation The SYMLINK variable specifies to mythicalLibrarian how to treat the files. There are three modes to SYMLINK. MOVE is the preferred mode.

  • SYMLINK=MOVE Allows XBMC to see the file and know when it has been removed from the library. When SYMLINK=MOVE and you have set up mythtv-setup to follow symlinks when deleting, MythTV's Library will function normally and XBMC will have access to all content.
  • SYMLINK=LINK Keeps the original MythTV file in it's place and creates a link for XBMC to read. This link is not maintained and the symlink will remain after the original file has been deleted
  • SYMLINK=Disabled Moves the file to the new location and does not provide a symlink to MythTV. On top of this, it will purge MythTV's database of all information and pictures relating to the video content. MythTV will have no memory of the recording. This mode can be used if you wish to delete files from XBMC.
Represents awareness of files between MythTV and XBMC

1.2 Use of mythicalLibrarian on MythTV Applications

This section covers setup of MythTV without any changes to mythicalLibrarian settings. This will create only a basic setup. Make sure to follow up by checking the /home/mythtv/mythicalLibrarian/output.log file after the first run.

sudo apt-get install curl agrep libnotify-bin
sudo mkdir /home/mythtv /home/mythtv/mythicalLibrarian /home/mythtv/Episodes /home/mythtv/Movies /home/mythtv/Videos /home/mythtv/FailSafe
sudo chown -R mythtv:mythtv /home/mythtv /home/mythtv/mythicalLibrarian /home/mythtv/Episodes /home/mythtv/Movies /home/mythtv/Videos /home/mythtv/FailSafe
sudo mv ~/Desktop/ /home/mythtv/mythicalLibrarian/
sudo mv ~/Desktop/librarian-notify-send /usr/local/bin/librarian-notify-send
sudo chmod +x /usr/local/bin/librarian-notify-send
sudo chmod +x /home/mythtv/mythicalLibrarian/
1.2.1 MythTV settings

See use of mythicalLibrarian on MythTV Applications for setting up mythtv here:

sudo adduser *Your User Name here* mythtv
  • Click System/Administration MythTV Backend setup
  • Select "1. General" then select next until you get to screen 3
  • Check "Follow symbolic links when deleting files".
  • Select next until you get to screen 8
  • Check "Allow user job #1 Jobs"
  • Select next until you get to screen 10
UserJob #1 Description: mythicalLibrarian
User Job #1 Command: /usr/local/bin/mythicalLibrarian "%DIR%/%FILE%"
  • Select Finish to exit
  • Select 6. Storage Directories
  • Select Default
  • Add a recording folder to which MythTV has access ie. /home/mythtv/Videos
  • Add additional folders if required.
  • Remove folders to which MythTV cannot run move jobs such as /lib or /var

1.3 Use of mythicalLibrarian on non-MythTV Applications

This section covers setup of non-MythTV applications. Make sure permissions are set correctly.

  • edit and set Database=Disabled
  • Set mythicalLibrarian up to run the job as follows:
/home/mythtv/mythicalLibrarian/ "Show name" "Show Title" "File to rename"
  • see Setting the Settings in this section

2 Using the command line

mythicalLibrarian has many command line switches. This section covers command line use of mythicalLibrarian.

2.1 mythicalLibrarian Normal Operation

This section covers normal operation of mythicalLibrarian When using mythicalLibrarian to rename files individually without database access, you can run the following command. Without database access, mythicalLibrarian can only look up episodes with title and subtitle.

mythicalLibrarian /home/mythtv/videos/1024_33020100321.mpg "spongebob squarepants" "giant squidward"
mythicalLibrarian /path/to/video/file.ext "episode title" "episode subtitle"

When using mythicalLibrarian with the mythtv database access you can run mythicalLibrarian as the following command.

mythicalLibrarian /home/mythtv/videos/1024_33020100321.mpg
mythicalLibrarian /path/to/mythtv/videos/filename.ext

mythicalLibrarian will integrate with the mythtv database as a user job after recordings. The mythtv user job for this is:

/usr/local/bin/mythicalLibrarian "%DIR%/%FILE%"
2.2 mythicalLibrarian --help

This section covers -h|--help

mythicalLibrarian --help

--help will display a help message containing helpful information

2.3 mythicalLibrarian --scan

This section covers -s|--scan

mythicalLibrarian --scan mpg /home/mythtv/Videos
mythicalLibrarian --scan ext /path/to/video/files

--scan will search for files automatically and process them. --scan has two parmaters. The first parameter is the extension of the files to search. The seccond parameter is the folder to search.

2.4 mythicalLibrarian --doover

This section covers -d|--doover

mythicalLibrarian --doover 

mythicalLibrarian keeps track of each file which fails while it runs. At any time you can type "mythicalLibrarian --doover" and all previously failed files will be processed. This process cannot be interrupted for any reason or data loss may occour.

2.5 mythicalLibrarian --undo

This section covers -u|--undo

mythicalLibrarian --undo

mythicalLibrarian keeps track of every file which has been moved. At any time you can type mythicalLibrarian --undo and all files will be reverted to their original state. There is no indication that --doover is running until it completes. This process cannot be interrupted or data loss may occour.

2.6 mythicalLibrarian --update

This section covers --update

sudo mythicalLibrarian --update

--update is a complicated process. First, mythicalLibrarian will download a copy of the mythicalSetup from the SVN repository. Next, it will download a copy of mythicalLibrarian from the SVN repository. mythicalSetup will then separate the program from the user settings. mythicalSetup perform several checks and bring up a GUI. The GUI will ask the user for preferences. mythicalSetup will attempt to set permissions and settings to appropriate defaults based upon user settings and the system. mythicalSetup will then combine the user settings with the separated program and place the file in /usr/local/bin. mythicalSetup will then call mythicalLibrarian --mythicalDiagnostics, which checks out your system. mythicalDiagnostics then calls mythcalLibrarian testfile.ext "mister" "episode 1332", which puts mythicalLibrarian into a diagnostic run. Once complete, the mythicalLibrarian in test mode, mythicalDiagnostics, mythicalSetup and update are closed and results are returned to the user.

Note: it is always a good idea to run mythicalLibrarian --doMaintenance after an update.

2.7 mythicalLibrarian --mythicalDiagnostics

This section covers -m|--mythicalDiagnostics

mythicalLibrarian --mythicalDiagnostics

--mythicalDiagnostics is a step-by-step functions check of mythicalLibrarian which will test functionality. It tests for all known causes of failure. This is a good troubleshooting tool for you to work the kinks out of your settings. mythicalLibrarian runs a series of diagnostics and then runs a search for an episode of mr rodgers with the file name set to testfile.ext. testfile.ext truncates the execution of mythicalLibrarian before any file operations occur. mythicalDiagnostics will return results which should look like this

  All tests complete            RESULTS
  Installed Packages:        ***PASSED*** 
  Notifications Test:        ***PASSED***
  XBMC Communications:       ***PASSED***
  File System Checks:        ***PASSED***
  mythicalTesting:           ***PASSED***
  Overall mythicalReadiness: ***PASSED***
-----End of diagnostics-----

Notifications test and xbmc communications test will not fail overall mythicalReadiness. Only critical checks effect overall mythicalReadiness. Critical checks are dependencies, file system, and mythicalTesting.

3 User Jobs

mythicalLibrarian allows flexibility in it's programming through the use of MythicalLibrarian UserJobs.

3.1 Benefeits of user jobs

User Jobs are supported to allow for your individual custom set-up to be personalized. You can rename files, and set custom commands to be run after execution of mythicalLibrarian. By using MythicalLibrarian UserJobs in mythicalLibrarian you can upgrade without loosing the work you have put into it.

Please post in the forums about any user jobs you have come up with. They will be of use to others.

4 Librarian Interfaces

mythicalLibrarian has the capability to send notifications to lib-notify on the DBUS of a Linux computer and keep your XBMC library up-to-date and clean. This section covers setup of notifications of the various platforms supported by mythicalLibrarian

4.1 XBMC Commands Another Sucessful library addition

This section covers setup of XBMC Commands. mythicalLibrarain can keep XBMC's library up to date, clean out deleted files and send a notification upon a completed job.

  • edit in your text editor
  • set XBMCUpdate, XBMCClean, and XBMCNotify to the desired settings, either Enabled, or Disabled
  • Set the IP and port under XBMCIPs. Separate IP addresses with a space. eg.
XBMCIPs=( " XBOX:8080" )

4.2 GNOME Desktop Notifications

Another Sucessful library addition
  • mythicalLibrarian requires no-password sudo access to send GNOME notifications.

Desktop notifications can be turned on and off from within the script settings. Desktop notifications take advantage of the lib-notify DBUS feature. DBUS is only supported on Linux.

Desktop notifications are used on the back-end machine by mythicalLibrarian to let you know that a file move operation is in process or an operation has completed. The external script librarian-notify-send becomes the specified username and sends a GNOME desktop notification. librarian-notify-send requires the user mythtv to have access to the sudo command without a password if run as a user job.

  • Get the current version of librarian-notify-send from Google Code download link
  • place the file in /usr/local/bin
  • add mythtvall group
sudo addgroup mythtvall
  • add mythtv to the mythtvall group
sudo adduser mythtv mythtvall
  • edit the sudoers file
sudo visudo
  • add the following line to the sudoers
  • hit ctrl-x and then y, then enter to write out and quit visudo
  • make librarian-notify-send executable by typing
sudo chmod +x /usr/local/bin/librarian-notify-send
  • edit mythicalLibrarian and set Notify=Enabled and NotifyUserName=*your user name*

You are done.

5 Troubleshooting

While mythicalLibrarian is designed to run fully autonomously, configuration issues come up from time to time. This section deals with these set-up issues

5.1 Random Errors

This section of Troubleshooting is designated for random errors which make no sense. These include errors which make you think mythicalLibrarian simply does not work.

5.1.1 Dependencies

This section covers dependencies of mythicalLibrarian. Usually you can find dependencies by typing "command -V" or "command --version"

  • The following are the versions of the binaries which mythicalLibrarian depends upon and are not in OS Karmic Koala.
agrep -V This is agrep version 3.0, 1994.
curl 7.19.5 (i486-pc-linux-gnu) libcurl/7.19.5 OpenSSL/0.9.8g zlib/ libidn/1.15
dialog Version: 1.1-20080819
mysql  Ver 14.14 Distrib 5.1.37, for debian-linux-gnu (i486) using EditLine wrapper
notify-send 0.4.5
  • The Following are the versions of the binaries which mythicalLibrarian depends upon and are included in OS Karmic Koala.
bash GNU bash, version 4.0.33(1)-release (i486-pc-linux-gnu)
(GNU coreutils) 7.4
sed GNU sed version 4.2.1
grep GNU grep 2.5.4
awk GNU Awk 3.1.6
ifconfig 1.42 (2001-04-13) net-tools 1.60
replace  Ver 1.4 for debian-linux-gnu at i486

5.2 Permission Errors

This section of Troubleshooting is designated for permission errors.

5.2.1 permission errors when run from terminal

Ensure the user running mythicalLibrarian has permission to run mythicalLibrarian. This includes read and write permissions to the working folder (default=~/mythicalLibrarian) see this post for more information:

5.2.2 mythicalLibrarian is not generating a log

Make sure is executable. You can set the permissions by typing "sudo chmod +x /path/to/" note: mythtv will be the user running mythicalLibrarian when run as a user job from MythTV see this post for more information:

remember, the user who launched mythtv-backend has to have permissions. You can check this manually by using the following:

sudo su *user who launched mythtv-backend*
mythicalLibrarian --mythicalDiagnostics

where *user who launched mythtv-backend* is the username of the user who launched mythtv-backend

5.2.3 Files are not being renamed or moved, but they are owned my mythtv

mythtv must have full access to the files. the /var/ folder does not allow users to move files without special permissions. Try moving your recording directory to your /home/ folder

5.2.4 mythicalLibrarian works from terminal, but not from user job

"mythtv" is the user calling the user job. If the user running mythicalLibrarian does not have access to the file, then mythicalLibrarian will not have access to the file. MythTV users can troubleshoot by changing your username with "sudo su mythtv", then running mythicalLibrarian.

5.2.5 mythcialLibrarian will not copy to my NFTF/FAT/EXT3 mounted drive

Additonal file systems may require editing the /etc/fstab file to change permissions. Search your OS documentation for information on how to mount a file system under a different username. Alternately, you may wish to add the user running mythicalLibrarian to your user group.

5.3 XBMC library is not updated

This section of Troubleshooting is designated for XBMC specific issues

5.3.1 XBMC does not show notifications

mythicalLibrarian requires the user to set the IP address of XBMC within

  • ensure you have a static IP set up in your router, see your router's manufacturer's website for more information
  • Open and edit the "XBMCIPs" setting to your XBMC IP.

5.3.2 XBMC shows notifications but Library is not updated

5.4 OS Speciffic issues

This section of Troubleshooting is designated for OS Specific issues

5.4.1 Gentoo

Gentoo OS requires upgrade of many packages. For sure you will need the package TRE. See here for version information:

5.5 Arch Linux

Arch Linux utilizes Python3 for it's primary Python interperater. Python 2.6 is used for MythDataGrabber(the program responsible for grabbing information from the MythTV database). You will need to install Python2.6

wget && tar xvf python26.tar.gz
cd python26 && makepkg -s
pacman -U python26-2.6.5-1-i686.pkg.tar.gz

After installing Python2.6, the MythDataGrabber shebang needs to be modified to specify the Python2.6 interperater. The first line in /usr/local/bin/MythDataGrabber should read

#! /usr/bin/python2.6

see here for more information

5.5.1 Mac OS X

If mythtv-backend is launched by a different user then yourself, then you will need to set permissions for that user. See here for more information:

5.6 Problems after upgrade to new version

This section of Troubleshooting is designated for issues after upgrade when the previous version was working

5.6.1 Episodes are not identifed

When episodes are not identified after an upgrade it is because the database handling has been upgraded, however, the database itself has not. Delete all database folders to correct the issue

5.6.2 mythicalLibrarian does not generate any files in it's working directory

Ensure the user running mythicalLibrarian has permission to run mythicalLibrarian. This includes read and write permissions to the working folder (default=~/mythicalLibrarian)

5.7 Episode recognition problems

5.7.1 Series title is not properly recognized does not have the proper Zap2it id included. Edit it. The program guide uses a generic name. Apply a showtranslation. See documentation included within

5.8 Episode title was not properly recognized is down. Wait for to resume and then run the file generated by mythicalLibrarian Ensure the data at TheTvDb is accurate. Correct if needed.

5.9 Desktop Notifications

This section of Troubleshooting is designated for GNOME Desktop Notifications issues

5.9.1 Desktop Notifications are not showing up

Ensure you have Notify=Enabled set in See this section for more information

5.10 Guide Data errors

for SchedulesDirect/Tribune/Zap2it set the GuideDataType setting in to "SchedulesDirect"

5.10.1 Currently Unsupported Guide data types

Support for guide data will be available with help from you. See this post:

5.11 Upgrade Issues

This section deals with issues relating to upgrading mythicalLibrarian

when upgrading mythicalLibrarian there may be random errors. When upgrading after more then a month it is a good idea to run

mythicalLibrarian --doMaintenance

When running do maintenance, you should ensure that all of your removable drives are attached. doMaintenance will flush the cache and mythicalLibrarian's internal database.

5.12 Misc

This section provides miscellanious fixes for user issues.

5.12.1 mythicalFileRestoration

When running in Symlink=Disabled mode, mythicalLibrarian deletes database information. In older versions of mythicalLibrarian all files were logged in the undo and doover que. When running in doover or undo mode, the files are first reverted to the original state and then reprocessed later. Because there is no database information, they cannot be reprocessed. However, all hope is not lost. The logs contain valuable information and Mike Szczys wrote a small Python application which will review these logs and restore the files.

NOTE: The script does check to make sure it's not overwriting existing files. However, it doesn't creat directories. You must manually create any subdirectories that don't already exist or the files will not be moved.


#! /usr/bin/python2.6
import os
myfile = "~/.mythicalLibrarian/"
f = open(myfile,'r')
for line in f:
   if line.startswith('test -f'):
       file1 = line.split(' && ')[0]
       file1 = file1.replace('test -f ',)
       file2 = line.split(' && ')[1].split(' ')[-1]
       if os.path.isfile(file2[1:-1]):
           if not os.path.isfile(file1[1:-1]):
              cmdLine = 'mv ' + file2 + ' ' + file1
              print cmdLine
              # uncomment the next line to activate the script 
              # print os.system(cmdLine)

Run this file first and view the output, if it works properly, uncomment the last line to activate it.

6 Notes

mythicalLibrarian is a spawn from the project BashSExx and MythSExx.

MythSExx and BashSExx have been depreciated.

Personal tools

Google Search
Wiki help