Designing Good Web APIs

I am not going to claim eons of experience in designing good APIs. I am going to approach it from the background of a developer who has to use and integrate with them.

What makes a good API? Here are some of my tenets:

1. Make your API calls as RESTful-like as possible

Because this term has evolved quite a bit I am going to stick with the primarily principles. Make API calls nouns describing "what" and not "do something".

2. API calls should have defaults.

Don't give me 100 required parameters just to see some action. Try as hard as you can to give me a basic call so I know I am doing something right. Along with this is to keep a minimum of API calls. The more calls you have documented the more I have to figure out which one it is I am suppose to be calling to get the right information. This is an area where I actually do want to be spoon-fed.

3. Use HTTP basic over SSL for authentication.

If you have the time and energy also support OAuth. The reason I hesitate with going straight to OAuth is because if you are just building an API your resources are probably limited. You are going to make mistakes. Getting it up and running with the lowest (and secure) common denominator is key.

Also use dedicated API keys that consist of an ID and a shared secret. Make sure these can be rolled.

4. Make your return format JSON.
For the love of god please don't use XML.

5. Version your API call in the end point:

/api/1.0/user/3
/api/2.0/user/3

6. Make the calls easily testable.

To me this means I can plug it into a web browser, type in some credentials, and get a response back. As a developer this makes me feel good early on.

7. Document your API calls with examples.

Especially the ones where I can click on a link and it does an API call for me. This is also great for testing your call.


A number of these guidelines are geared heavily to web development. If you are designing an API for a message passing system with latency and size requirements a good deal of this would change.

How to get a 2yr single root SSL certificate for $10

I recently had to do some shopping for a SSL certificate for my startup Stratismo (I linked to it incase anyone wants to see the actual certificate). Obviously I wanted one that was widely supported and didn't really want to pay a ridiculous amount for it.

So shopping around I landed on RapidSSL's site. RapidSSL has a program where you can switch to them if you have a competitor's certificate. On top of that they also give you a free year. All of this for free. The program details are here:

http://www.rapidssl.com/switch-ssl/index.html


I just happened to have a Comodo certificate so I gave this a test run. The Comodo certificate had some issues because of the number of intermediate certificates I had to include. While this shouldn't cause problems in theory in reality it is just one more thing to deal with.

So what I did was apply for RapidSSL. They ended up giving me one of their SSL certificates for free after verifying I had a Comodo SSL certificate. They also extended the validation by a year so basically I got two years for $10. RapidSSL certificates are also signed by Equifax which is installed on almost every browser.

Overall a great deal.

PySAML - A SAML 2.0 Toolkit for Python

Security Assertion Markup Language (SAML) can be a bit confusing to understand. At its core SAML is just a protocol with defined messages written in XML. The main purpose of SAML is to enable you to log in at one place such as a website and then jump over to another website without having to log in again. This setup is commonly called "federation".

It is similiar to OAuth. I see SAML continuing to make headroom in the enterprise space while OAuth stays strong in the consumer space. However in the future I hope these two technologies will end up playing well together.

In either case I am releasing a small Python library for generating SAML assertions. The main purpose of this is to learn SAML by doing (actually having to create an assertion gives me a good idea of the complexity of the protocol).

The library depends on M2Crypto so download and install it:
http://chandlerproject.org/bin/view/Projects/MeTooCrypto

M2Crypto depends on SWIG so you might need that as well.

Once that is installed you can download my distribution of PySAML here:

Python 2.6 - Win32:
http://darkeneddesire.com/PySAML/PySAML-1.0.win32.exe

Unix (source):
http://darkeneddesire.com/PySAML/PySAML.tar.gz

For windows you just run the executable. For unix you should run the following commands:

# cd PySAML
# python setup.py build
# python setup.by install

There are examples in the "examples" folder if you download the full source.

The whole project is available on github as well:
http://github.com/tachang/PySAML/tree/master

Trying to get Dual-SIM to work in T-Mobile G1

There is a product being sold called "Magic SIM".

http://www.magicsim.com/en/dual_sim_show.asp?id=49&cp_id=&sort2name=14

Basically it is a SIM card that has two slots and a small memory chip. After physically cutting up your two SIM cards and putting them into the slots you insert the whole thing into your phone.

If your phone supports the GSM STK (SIM Tool Kit) standard then you can switch back and forth between the two SIM cards.

However for your phone to actually support the standard the manufacturer had to have built it into the operating system as well as provide a tool to "talk" using the STK.

I currently see that the Android development tree has some Stk development going on. I downloaded a precompiled Stk.apk and installed it on my phone with the card. However it does not seem to work.

While I do see the SIM Toolkit menu icon and it does seem to read from the custom card I am unable to obtain a carrier signal from either of my SIM cards. When I scroll to any of the menu options such as "SIM 1" or "SIM 2" and press the trackball the phone simply gives me a rotating circle on the upper right saying it is busy. And at that point it hangs.

Here is a quick and dirty image of my phone as well as what the SIM menu looks like:

Using Github Through Draconian Proxies (Windows And Unix)

Here is a pretty standard scenario at most corporations:

- All access to the internet is restricted to a proxy
- The proxy only allows connections out on port 80 and 443
- CONNECT method is only enabled for 443
- Proxy Authentication is required (NTLM or Basic)

I like to use both Windows and Unix environments. On Unix tunneling to Github is a bit easier because lots of tools are included.

Unix

1. Download Git. At the time I was writing this I was using Ubuntu so I simply did apt-get install git-core

2. Download and install corkscrew (http://www.agroman.net/corkscrew/). This is a tool for tunneling SSH through HTTP proxies.

3. Edit or create the file ~/.ssh/config and put the following:

ProxyCommand /usr/bin/corkscrew proxy.example.com 443 %h %p ~/.ssh/myauth

Host github.com
User git
Port 22
Hostname github.com
IdentityFile "/media/truecrypt1/Keys/GitHubKey.private"
TCPKeepAlive yes
IdentitiesOnly yes

Host ssh.github.com
User git
Port 443
Hostname ssh.github.com
IdentityFile "/media/truecrypt1/Keys/GitHubKey.private"
TCPKeepAlive yes
IdentitiesOnly yes

• The ProxyCommand is invoked when ssh needs to make a connection. We are telling ssh to use /usr/bin/corkscrew. This is a 3rd party program that sets up a socket through the HTTP proxy.
• The program /usr/bin/corkscrew takes as its 5th argument a file containing credentials for your HTTP proxy. Not all proxies need authentication but if you do just put in the file a single line formatted username:password.
• The Host github.com indicates to ssh that if we are connecting to github.com to use these specific settings. There is nothing special here except we specify the location of the private key that corresponds to the public key we had over in http://www.github.com/
• Notice we have another entry titled "Host ssh.github.com" . This is to get around proxies that only allow the CONNECT command over 443 (the truly locked down ones). To get around this github setup a whole separate host that listens on port 443. We add both entries here since they are both valid.
  

4. If everything is setup correctly you should be able to run:
# ssh github.com

Hi tachang! You've successfully authenticated, but GitHub does not provide shell access.
Connection to github.com closed.

If this doesn't work you can run
# ssh ssh.github.com

And get the exact same thing. If the first command didn't work it means you are using a proxy that blocks CONNECT on port 22. Almost no proxies block CONNECT on port 443 because you need that for SSL.


We get a no shell access message from github because we are trying to obtain a shell and github has it disabled. However this indicates everything is working. This concludes the setup for Unix.

------------------------

Windows


1. Download msysgit http://code.google.com/p/msysgit/

Some settings:

- "Run Git from the Windows Command Prompt"
- "Use OpenSSH" (this one is very important)
- Pick your line endings

2. Download connect.c
http://bent.latency.net/bent/darcs/goto-san-connect-1.85/src/connect.html

This tool deserves its own post mostly because of its utter simplicity. It mirrors the open source tool corkscrew and is used for tunneling through proxies. Yes the tool's name is really called "connect.c".

For Window's users, a pre-compiled binary is available:
connect.exe
I put my connect.exe in C:\Windows\connect.exe

3. Decide whether you like to use the Windows cmd.exe to do stuff or the Cygwin style shell. Or both.

Cygwin Git Bash Shell
For the Cygwin style shell start up the Git icon and edit the file ~/.ssh/config
*Make sure the file has no extension.

Put the following in that file:

ProxyCommand /c/windows/connect.exe -H username@proxy.example.com:443 %h %p

Host github.com
User git
Port 22
Hostname github.com
IdentityFile "/c/Keys/GitHubKey.private"
TCPKeepAlive yes
IdentitiesOnly yes

Host ssh.github.com
User git
Port 443
Hostname ssh.github.com
IdentityFile "/c/Keys/GitHubKey.private"
TCPKeepAlive yes
IdentitiesOnly yes

• Notice the slash style in order to access the file system.
  
• The proxy username is specified as part of the proxy setting. The password for the proxy is prompted for. Read more about connect.c to figure out how to get rid of this prompt.

At this point, using the Git Bash shell should yield:

$ ssh github.com

Hi tachang! You've successfully authenticated, but GitHub does not provide shell access.
Connection to github.com closed


Windows cmd.exe shell

Suppose you don't like the Git Bash shell. You prefer the cmd.exe interpreter.

- Go to your config file at C:\Documents and Settings\\.ssh\config
- Make a copy of it or make a new one. I called mine config-windows

Put the following in the file:


ProxyCommand C:/Windows/connect.exe -H username@proxy.example.com:443 %h %p

Host github.com
User git
Port 22
Hostname github.com
IdentityFile "C:\Keys\GitHubKey.private"
TCPKeepAlive yes
IdentitiesOnly yes

Host ssh.github.com
User git
Port 443
Hostname ssh.github.com
IdentityFile "C:\Keys\GitHubKey.private"
TCPKeepAlive yes
IdentitiesOnly yes

• Notice the mixture of slash styles. I find this rather odd but it is what works. We have a forward slash style for the ProxyCommand but for the IdentityFile a forward slash or backward slash both work.

Running the command (making sure we run Git\bin's ssh.exe and not some other one in the PATH):

C:\Program Files\Git\bin>ssh.exe -F "C:\Documents and Settings\\.ssh\config-windows" github.com

Hi tachang! You've successfully authenticated, but GitHub does not provide shell access.
Connection to github.com closed


General Git Cloning

- Make sure you are using the right Git URL:

Suppose your Public Clone URL is: git://github.com/tachang/EyeFiServer.git

You should use the following URL that utilizes the SSH transport:

git clone ssh://git@github.com:22/tachang/EyeFiServer.git
git clone ssh://git@ssh.github.com:443/tachang/EyeFiServer.git

Django TypeError Exception

One of the central pieces of Django is urls.py:

http://docs.djangoproject.com/en/dev/topics/http/urls/

This file is well documented and serves as a central place to match what pages get served by what URLs. Seems simple enough since they accept regular expressions.

One thing I did run into though was getting a TypeError exception. It looked something like this:

TypeError at /applicationname/

index() takes exactly 1 argument (2 given)

To me I was puzzled why this was happening. The reason is that when Django parses the regular expression it not only does a match but saves each of the tuples in the regular expression. In the documentation it is this line:

"The view gets passed an HttpRequest as its first argument and any values captured in the regex as remaining arguments."

So what happens is that when you start to get fancy with your regular expressions the function signature in your views.py needs to change as well to accommodate the extra parameters.

SiteMinder R12 Admin GUI

If you are upgrading or installing SiteMinder R12 you may have noticed that the traditional admin GUI has changed. The Java applet now asks you for 4 fields.

Username: This is the username found in the "Administrators" list on the old version 6 GUI. The applet is not case sensitive.
Password: The password in the "Administrators" list.
Host Name: A 4.x compatible Agent's name
Passphrase: The Shared Secret of the 4.x compatible agent

The Host Name and Passphrase fields are where things get interesting. SiteMinder R12 is trying heavily to move the functions of this applet to a web based system. The web based system communicates with the SiteMinder policy server using the API. However, the API still uses 4.x agents as the method of authentication to the policy server. 4.x agents have an associated shared secret.

Thus to get into the traditional 6.0 GUI you need to have a 4.x compatible agent. But what happens when you upgrade to SiteMinder R12 and don't have one? You have two options:

1. Install the SiteMinder Web Access Manager GUI - Unfortunately the installer for Windows is over 2GB
2. Manually create a 4.x Web Agent

Option 2 is easiest. We are going to create a file with the proper parameters and then use smobjimport to import it directly to the policy store. To do this create a file called "Generic4xAgent" and put the following in the file:

objectclass: Agent
Oid: 01-39c83ef9-5c51-4fb4-ba13-193543b8a9d4
Name: siteminder
Desc:
AgentType: 10-8d78bb96-ae15-11d1-9cdd-006008aac24b
RealmHintAttrId: 0

objectclass: TrustedHost
Oid: 24-5ea55269-d8a9-47a0-864c-c97026c00b99
Name: siteminder
Desc:
IpAddr: 127.0.0.1
Secret: password
Is4xHost: true
RolloverEnabled: false
SecretGenTime: 00000000-00000000-000000000000000000000000000000000000000000000000
SecretUsedTime: 00000000-00000000-000000000000000000000000000000000000000000000000
PrevSecret:

Save the file as "Generic4xAgent". Now run the command:
smobjimport -iGeneric4xAgent -dsiteminder -wpassword -c

-i is the filename
-d is the SiteMinder admin username
-w is the SiteMinder admin password
-c indicates that the file has cleartext passwords

This creates a 4.x agent named "siteminder" with a shared secret of "password".

At this point you should be able to login to the traditional admin UI.