Monthly Archives: August 2008

Computers as Infallible in Pulp Culture

Posted on by
Reply

I was watching an old episode of Star Trek the original series the other day and was intrigued that they portrayed computers as infallible, pure logic devices. In the episode, Kirk’s on trial for his life/command because a computer record shows him making a mistake. Throughout the episode there are numerous comments such as “the computer cannot be wrong” and “if the computer said you did it, it must have happened”. You can also see other such representations in old Twilight Zone episodes and movies.

A few questions come to mind:

  • Was the basis for believing computers to be infallible based on fact or fiction?
  • Were computers significantly more stable than those today?
  • Is the belief of computers being infallible still held today?

For the first question, I can find some factual basis for this belief, namely the existence of logical programming languages such as Prolog and later-day semantic programming languages. In such systems, you program a set of ‘facts’ and ‘relations’ and the system using them to build a knowledge basis out of what it knows. Given the large search space and computational complexity of such systems, the work is still more theoretical in nature and rarely used in modern day software systems. Logical ideas may be a component of some systems, but the actual code is written in more expressive languages like Java, C++, PHP, etc. The question still remains whether these earlier authors know about logical programming, or really just made up the idea that computers are infallible (or will be in the future!) to fit their scenario. It would be interesting to interview the TV/movies writers from this era that are still around to determine where these notions came from.

For the question of stability, I would say yes, computers back in the day were probably a lot more stable in part because the complexity of such systems was extremely low. Also there wasn’t the notion of having separate hardware, operating systems, and applications; in many cases the companies sold the devices as one piece (much in the way Apple still does), so that reliability was much easier to control and maintain. These days accessing your bank records via an online website may involve dozens of systems all running general purpose hardware and operating systems, any of which can easily fail or make mistakes if not properly configured and managed. It’s like trying to hook up a Windows-only digital camera to a Mac only on a much larger scale.

The last question of whether or not computers are seen as infallible today remains a question open for wide debate. As a software programmer I have an interesting perspective on the topic, in that I know what can go wrong in systems. Most people who listen to the news know that their information may not be safe (see credit card theft or even just lost laptops with social security numbers on them), but the data can be corrupted from far less malicious problems like concurrency and device failure. That’s why most (but not all) systems use fault tolerant solutions; if a computer or hard drive fails, the system can detect and recover from this failure. Unfortunately for anyone who’s ever been affected by a computer glitch in one of their accounts, myself included, calling for customer support can be daunting. Representatives are trained only to believe what the screen in front of them tells them and will rarely take your word for it should you argue the computer is wrong. I can recall one conversation with a company that deleted my 3 year old account and then claimed that I never had an account with them. The only thing that finally convinced the representative to believe me was that while the computer records showed I had no account and never had one, it also said I had a history of support calls about my account. Once they realized the inherent contradiction of me making phone calls for 3 years about a non-existent account they started to believe me, but prior to that point treated me as if I was making things up. I’ve heard other horror stories from the field of other customer support representatives (Verizon: for example) who, when faced with such a contradiction, will stick to whatever the computer screen says, even when common sense dictates it is wrong.

As most people know I am a daily reader and advocate of The Daily WTF which posts real stories (many of which you can verify) about what goes wrong in the world of technology. Based on their posts, it paints the picture of an industry which often holds software together with glue and bits of string.

As we move forward with increasingly complicated software systems (J2EE, .NET, Flex), it seems inevitable that computer systems will be less and less stable. If so, I can only take comfort in the idea that perhaps the public will be more aware how fragile systems are and start trusting individuals over computers. It’s often been said computers can only do what you tell them to do, and if you have developers who don’t fully understand the user experience, you end up with systems that do not work properly. On the other end of the spectrum, there’s increased reliance on using computers for passports, face detection (especially in government terrorist detection systems), and a myriad of bio-related information systems. In that regard, good luck trying to get a plane when the computer has incorrectly invalidated your passport.

Note: One thing I left out of this discussion is the topic of security. Computers today are far less stable than they were originally perceived to be. For example, in that same Star Trek episode it is discovered that someone altered the computer records to frame the captain and that only a handful of people on the ship were capable of such an action. We now know even a 13 year-old is capable of hacking large systems, but that is in part based on the inherent insecure nature of many of the systems, especially the Internet. I see security as a planning and deployment problem, one capable of being solved if the developers spend the time to address it (which they rarely do). In regards to the growth of the Internet, the protocols that won out were the easiest to setup and maintain, not the ones that were the most safe. With the increased bandwidth we all have now (cable/phone modems versus 1200 baud modems), it is my hope that a truly secure Internet will become available down the road.

Question mark ‘?’ characters as text in JDBC

Posted on by
1

Many people wonder how insert strings containing question mark characters into their database queries via JDBC. For those unfamiliar with the problem, ? is a reserved character in JDBC queries for parameterizing inputs. For example, if you have run the same query searching for a user but each time with a different name, JDBC offers you the ability to precompile and save the parameterized form of the query with ?’s, thereby saving the overhead of creating lots of new database statements. First, let’s frame the problem. Consider the following code:

PreparedStatement ps = conn.prepareStatement("SELECT * FROM widgets WHERE note LIKE '%long?'");
ResultSet rs = ps.executeQuery();

Description: This code searches for all widgets with note field that ends in the phrase “long?”, such as “This is how long?”.

Your first thought might be why make this a PreparedStatement at all (which supports parameters), you could just as easily do it with a Statement (which does not support parameters). Under most circumstances, it is a good coding practice to use PreparedStatement over Statements, even if you don’t have any input parameters. It allows you add parameters easily such as:

PreparedStatement ps = conn.prepareStatement("SELECT * FROM widgets WHERE size > ? AND description LIKE '%long?'");
ps.setInt(1,100);
ResultSet rs = ps.executeQuery();

Question: Will this code compile and run?

The answer is that it will compile, but under most circumstances (depending on the JDBC driver) it will not run. Why? The answer highlights just how dumb JDBC drivers really are. In short, they don’t really understand anything about the data they are parsing other than “I see a question mark, let me replace it with something!”. In this example, the user replaced the first ? with an integer, but did not replace the second question mark. In this regard, the JDBC driver will throw a runtime exception prior to sending the code to the database. Also note this code will have the same problem whether you are inserting the value ‘%long?’ into the database or reading it; as I said the JDBC driver knows very little about the query you’re constructing other than its find and replace mentality.

There’s a number of solutions available although my favorite is the following:

PreparedStatement ps = conn.prepareStatement("SELECT * FROM widgets WHERE size > ? AND description LIKE ?");
ps.setInt(1,100);
ps.setString(2,"%long?");
ResultSet rs = ps.executeQuery();

Notice I don’t need the single quotes around the parameter, JDBC will do this for me. This is better not just because it solves our original problem, the code will now run, but we’ve parameterized a messy string query! Solving the problem and enforcing good code practices is a win-win. What might throw you for a loop is you’ve increased the number of question mark ?s in the code by one. Whereas before the second ? in the query was a character representing text to be searched on, the second ? now represents a parameter JDBC should replace. It could be replaced with our target string ‘%long?’ or something without a question mark at all such as ‘horse’. Part of the advantage of parameterizing your inputs in the first place is you don’t have to worry about such situations if a user enters a question mark as a value.

Transferring Tivo episodes to your iPhone with a Mac and PC

Posted on by
1

If you’re like me and you own a Tivo HD/Tivo Series3 and an Apple iPhone/iTouch/iPod, you may be wondering, can I transfer shows from my Tivo to my portable device? The answer is yes, although the path you choose depends a lot about your home setup. For example, I have a MacBook and a PC. The MacBook is what I use to sync to my iPhone, but laptops have limited space, so I use my PC to transfer the gigabytes of data from my Tivo. Combining the two work together can be a daunting task so I’ve written this guide of my experience to help people in similar circumstances. Once set up the results are spectacular, gigabytes of data are transferred from my Tivo to my PC, automatically converted to Apple iPod format, and the most recent episodes are automatically transferred to my iPhone, whenever I sync my iPhone to my Mac.

Part 1: Setting up your PC for the Tivo Transfer

The Tivo and PC part of the equation is pretty straight-forward even if the Apple/iTunes can be a bit more challenging. First off, you need a PC to download and convert the files to work on your iPod. For whatever reason, Tivo has developed the PC version with many amazing features, but has yet to put them in the Mac version. They do suggest Roxio Toast Titanium but I’ll leave it to someone else to write a guide for this software. Back to the PC version… for a one-time fee of $24.95 you can purchase an unlock key of Tivo Desktop Plus for PC. Once installed and configured with a Media Key for your specific Tivo, you can select “Auto-Transfers”, select shows you’ve previously records, and have it download new episodes of shows automatically. Its equivalent to a download-form of a Tivo ‘season pass’. You can then open Tivo Desktop and go to Preferences -> Portable Devices and set it to convert it to Apple iPod as well as delete the original larger file. This saves a ton of space since it only keeps the final, converted item.

Some reasons why I use a PC for the transfer? My PC has terabytes of data, my MacBook does not. So even if they had a native Mac version, I’d still use my pc.

Part 2: Setting up your Mac for the iTunes Import

If you followed part one you now have a single folder filled with Apple-formatted shows. If you’re like me you set them on a read-only network share for other computers in your home to use (and open the Keychain Application on a Mac to set the share to no longer prompt for a username/password). Now we move on to the Mac side of things…

While I like my PC for the transfer and storage part, I prefer my Mac to sync my iPhone. If you go into iTunes preferences -> Advanced, there’s an option “Copy files to your library when adding to Library”. For the purposes of this explanation we’re going to assume you turn this off so that when you add a file from the external share, only a reference is copied. The file stays on the network share. With this setting turned off you can open iTunes and go to File -> Add Folder to Library, select the network share, and it will add all the files in the folder to your iTunes library. But there’s a few catches:

1) This process is not automated, future files will not automatically be detected and added
2) Sometimes iTunes screws up and lists files twice
3) Some shows (like daily show) requires name changes because it saves the name of *every episode* as “The Daily Show with …” making it impossible to read on you iPhone.

For that I’ve written an Apple script that fixes all the problem above (see appendix). I set it on a cron job to occur every 15 minutes (search google for instructions on using/running crontab on a Mac), first checking to make sure iTunes is open and the network share exists. If all this is in place, you’ll have your iTunes library constantly updated any time it is at home, the network is connected, and iTunes is up. I also set up a separate apple script to reconnect the network drive on computer startup (see appendix).

It takes a little practice but AppleScript is an extremely simple language to work with. After you download the script, read it over and learn what each line means. This way you can modify it to fit your custom situation as needed. I take no responsibility if this script, or your modified version of it, does anything bad to your computer or your files. Use at your own risk.

Part 3: Setting up your iPhone

Finally, the ‘easy’ part. With all this set up, your Mac will now contain references to recently converted shows (usually delayed about 45 mins from actual airtime completion). If you leave your pc and Mac on all the time, this will happen while ya sleep.

The final part is to configure your iPhone, of which I do have some advice. For starters, iTunes is a tad novice-level in that you can either choose to transfer a playlist of tv video files XOR transfer recent episodes of tv shows. The exclusive “XOR” is because you cannot do both (as I said, the Apple/iTunes side is more of a pain). For simplicity, I like transfer the “3 most recent episodes” of “selected tv shows”. The only thing that can screw this up, really, is if Tivo suggestions records like 10 episodes of a show you didn’t want it to. For example, Comedy Central likes to air the Daily Show 5-10 times a day, so its possible, albeit unlikely for your to have multiple copies of the same episode on you iPhone (if you want to disable it all together just rate the Daily Show with 3 thumbs down or turn Tivo Suggestions off).

The result? Every morning I connect my iPhone to my Mac and it instantly downloads all of last nights programming including new episodes of daily show. I can then watch them on my commute. All of the setup is a little painful but the PC never needs to be touched once your auto-transfers are set up. The Mac is a little more unfriendly, in that you may have to run the script manually if you are in a hurry and forgot to leave your computer on over night.

Appendix: AppleScript code

Applescript to load share automatically on computer startup. You can use System Preferences auto-start to make it startup.

tell application "Finder"
open location "smb://ip-address-for-pc/my-ipod-share"
end tell

AppleScript to import and cleanup Tivo Files. Also available here: tivo-script.txt

-- syncPcTivoToApple v 1.0 - An AppleScript to automatically import a network
-- share of PC transferred iTune files to your Mac.
--
-- Copyright 2011 Scott Selikoff

-- Video Format
property format_items : {"MPG4"}

-- File extensions
property ext_items : {"mp4"}

-- Whether or not to perform import
property okflag : false

-- List of 'special' shows
property convertNameToDateRecordedIdentifiers : {"The Daily Show With Jon Stewart", "The Colbert Report"}

on run

-- Set name of network share via the file system
set network_folder to "my-ipod-share"

-- Proceed if network share exists and iTunes is open
tell application "Finder"
set okflag to ((exists folder network_folder) and ((get name of every process) contains "iTunes"))
end tell

if okflag then
import_files_to_itunes(network_folder)
remove_duplicates("TV Shows")
clean_track_name("TV Shows")
end if
end run

-- Perform Import
on import_files_to_itunes(this_folder)
set these_items to list folder this_folder without invisibles
repeat with i from 1 to the count of these_items
set this_item to alias ((this_folder as text) & ":" & (item i of these_items))
set the item_info to info for this_item

if (alias of the item_info is false) and
((the file type of the item_info is in the format_items) or
the name extension of the item_info is in the ext_items) then
tell application "iTunes"
add this_item to playlist "Library" of source "Library"
end tell
end if
end repeat
end import_files_to_itunes

-- Cleanup library of diplicates
on remove_duplicates(this_playlist_name)
tell application "iTunes"
set this_playlist to user playlist this_playlist_name
set all_tracks to the number of this_playlist's tracks
set temp1 to {}
set delete_list to {}

if all_tracks > 1 then
set this_location to the location of track 1 of this_playlist
repeat with i from 2 to all_tracks
set next_location to the location of track i of this_playlist

if this_location is equal to next_location then
copy i to end of delete_list -- then this track is a dupe; copy its index to our delete_list list
end if

set this_location to next_location
end repeat

-- total number of tracks to nix for dialog
set to_nix to delete_list's length

--If you must delete, do it backwards, ie:
repeat with x from to_nix to 1 by -1
copy (item x of delete_list) to j
delete file track j of this_playlist
end repeat
end if
end tell
end remove_duplicates

-- Cleanup special tracks
on clean_track_name(this_playlist_name)
tell application "iTunes"
set this_playlist to user playlist this_playlist_name
set all_tracks to the number of this_playlist's tracks

repeat with i from 1 to all_tracks
set artistName to the artist of track i of this_playlist
set titleName to the name of track i of this_playlist

if ((titleName begins with artistName) and (titleName contains "\"")) then
if (artistName is in convertNameToDateRecordedIdentifiers) then
-- Case 1:  Daily Show and Colbert Report, just use date as name
set pos1 to ((length of artistName) + 2)
set pos2 to ((length of artistName) + 9)
else
-- Case 2: All other shows strip out the artist name and date
set pos1 to ((length of artistName) + 12)
set pos2 to ((length of titleName) - 1)
end if

-- Position invariants must quality for renaming
if ((pos1 < pos2) and (pos1 < (length of titleName)) and (pos2 > 1)) then
set newTitleName to (text items pos1 thru pos2 of titleName) as text
set name of track i of this_playlist to newTitleName
end if
end if
end repeat
end tell
end clean_track_name