Web Security, Privacy and Commerce, 2nd Edition

A.4 Commercial Start-Up

Now that the Unix workstation was on the island and the leased line to the Internet was up and running, the next thing to do was to work on our dialup access.

A.4.1 Working with the Phone Company

A friend who ran an ISP in Cincinnati had told me that if I wanted to run a successful dialup operation, I should get a service from the phone company called circular hunting. Normally, a bank of telephones is put into what is called a hunt group. You might have a block of phone numbers, from 555-1000 to 555-1020. With normal hunting, a phone call to 555-1000 is always taken by the first phone in the hunt group that isn't busy. But with circular hunting, the phone system remembers the last phone that it dialed and automatically dials the next phone number in the hunt group, whether the call to the previous phone number has hung up or not.

Circular hunting sounded like a great idea if you are running dialup access with analog modems. Consider what happens if you have a modem that suddenly fails to answer new calls. If you have circular hunting, then you only lose one modem: the next caller gets the next modem. But if you don't have circular hunting, then every caller will get the ringing modem, and nobody will get any of the other modems in the hunt group that are still good.

A.4.1.1 Lesson: Design your systems to fail gracefully.

I called NYNEX and tried to order a Centrex system with circular hunting. Unfortunately, nobody at the phone company knew what I was talking about. (A few months later, I learned that the reason nobody knew what I was talking about was that the service has a different name in Massachusetts from the one it has in Ohio. In Massachusetts, the service is called UCD, Uniform Call Distribution.)

A.4.1.2 Lesson: Know your phone company. Know its terminology, the right contact people, the phone numbers for internal organizations, and everything else you can find out.

I ordered a conventional Centrex system with four lines. Three of the lines, 696-6650, 696-6651, and 696-6652, would be in the hunt group. The fourth line, 696-6653, would not be in the hunt group. That line would be our business line.

A.4.2 Incorporating Vineyard.NET

In mid-August, the Internet cooperative got a third partner: Bill Bennett. Bill had been watching everything that Eric and I had been doing and he wanted a piece of the action. I also owed Bill a tremendous amount of money, because the wiring of the house had cost far more money than I had budgeted. Bill was willing to forgive the loan in exchange for a percentage of the Internet cooperative.

It was quickly becoming clear that running the Internet access provider as a cooperative wasn't going to work in the long run. Unless we could make a profit, there would never be money to purchase new equipment and expand our capacity. Unless we could make a profit to hire somebody else, I would be stuck forever doing technical support. Bill thought that an aggressive commercial service could make a tremendous amount of money. Egged on in part by Bill, in part by my debts, and in part by a spate of Internet-related companies that had initial public offerings in the spring and summer of 1995 (at somewhat obscene valuations), the three of us incorporated Vineyard.NET and embarked on a slightly more aggressive service offering.

Our plans for offering service mimicked many other Internet companies that were starting at the time. We planned to let early adopters use our service free for a few months. Then we hoped to charge a low introductory rate, after which we hoped to raise our prices once again to the actual level.

A.4.3 Initial Expansion

The first things that we needed were more phone lines and modems. That required working again with NYNEX or, in our case, our NYNEX-authorized reseller. We told them that we wanted to have a fiber optic circuit brought from the central office to our location. But NYNEX wouldn't do it: they said that the fiber demultiplexing equipment was not CPE customer premise equipment. Instead, they brought a cable with 100 pairs of copper to our location. Bringing it required two huge trucks, five men, and shutting down the street for a day. We calculated that the whole operation must have cost NYNEX somewhere between $5,000 and $10,000. All of a sudden, things were getting serious. NYNEX had spent real money in the anticipation that we would be paying our bills. And to do that, we needed to get customers and collect money from them.

I knew that one of the most expensive things for a technology-based company to do is offer technical support to its customers. Tech support is even more expensive than research and development, since research and development costs remain roughly constant while tech support requirements increase along with a company's customer base. Another thing that's incredibly expensive is advertising. Rather than build our own technical support group, we partnered with computer stores on the island. They could sign people up for our Internet service when customers bought computers or came in to buy supplies. It seemed like a win-win situation.

A.4.3.1 Lesson: Build sensible business partnerships.

The idea of partnering made a lot of sense. The island's computer stores, after all, were already experienced in dealing with computer users on the island the people who would be our customers. And they were also equipped to sell customers any additional hardware or software that they might need to make their computers Internet-capable. We set up our systems so that our computer store partners would be able to create accounts on our machine. They would also collect sign-up fees. In return, they would get a bounty for each user they brought in, and a set percentage of each user's monthly fee. We also set up a few of the island's computer consultants as resellers.

Once we had our phone lines installed, we needed to figure out what to use for modems. We briefly looked at some rack-mounted modems made by U.S. Robotics and were scared away by the high prices. Although I wanted to use rack-mounted modems, it seemed that all we would be able to afford for a while would be discrete ones.

But which discrete modems? I bought some ZyXEL modems for a lot of money and they were having problems, so we started trying other brands. We settled on Motorola's Lifestyle 28.8 modems. They seemed to work reliably and they didn't give off much heat. Eric built a modem "rack" for them out of wood, with each modem tilted at a 45-degree angle so that the heat would vent out the back side. (Eventually, we switched over to rack-mounted modems manufactured by Microcom.)

We started offering service for free in August 1995. Our plans were for "charter" members people who signed up before October 1, 1995 to be charged $20/month for the first year. And then anybody who signed up in November would be charged $25/month. We wanted to keep our prices lower than $29/month that's what The Internet Access Company (TIAC) was charging. TIAC offered dialup access on Martha's Vineyard, and it was important to Eric that we charge less than they did.

A.4.4 Accounting Software

The next thing we realized was that we would need to have some sophisticated software for keeping track of user accounts.

It wasn't my intention to write a complete customer billing and accounting system in Perl. I really only wanted to have a system for keeping track of who had paid their monthly bills and who hadn't. I wanted customers to be able to check their balances from the Web. And I wanted to be able to send customers their bills by email.

I had run a business before, back in 1992, and had used QuickBooks to keep track of the business books. QuickBooks is made by Intuit, the makers of Quicken. QuickBooks can easily keep track of a customer-driven business with hundreds or even thousands of customers. But QuickBooks didn't have any easy way of importing lists of invoices that we might generate on the Unix system, and it didn't have any easy way of exporting the data for view on the Web. In the end, I used QuickBooks for the business's main books, but had to create my own system for managing user accounts.

It turned out that writing our own accounts management system was the right idea: it gave us the power to tailor our business policies and terms however we wished, knowing that we could easily modify our billing software to accommodate our desires.

For instance, we wanted our resellers to be paid a 20 percent commission on the accounts that they owned, but only when their customers actually paid their bills. That wasn't a problem: I simply modified the program that received payment so that when a check was received on a customer's account, the reseller was automatically credited with the commission.

A.4.4.1 Lesson: Make sure your programs are table-driven as often as possible.

From speaking with my friend in Cincinnati, I realized that we might have dozens of different kinds of accounts and special deals within a few months. It had become an accounting nightmare for him. Rather than repeat his experience of building this logic directly into our accounting system, I created an accounting system that was table-driven. Each customer had a different account type. The account type keyed into a database that included information such as the account's sign-up fee, its monthly fee, the number of hours included in that monthly fee, and the cost for each additional hour.

A.4.4.2 Lesson: Tailor your products for your customers.

We also created a special kind of account for small businesses called a "group" account. This account allowed a business to have several Internet accounts that would all be charged to a single bill. Businesses were charged on a different scale from residential users a lower monthly fee, but a higher hourly rate. We did this because many businesses seem more comfortable with a pay-as-you-go approach. (Or perhaps it's because businesses find it easier to let these charges creep up when they are not paying attention.) At any rate, going after business users made sense, because they had a peak usage time between 9 a.m. and 5 p.m., and the peak residential usage time was between 5 p.m. and 12 p.m.

We did not funnel the group accounts through our resellers. Instead, we resolved that we would provide tech support to a single person at each business; this person, in turn, was expected to provide first-line technical support to the other members of his organization. Once again, having built our own account management and billing software made this easy to do it was just a matter of coding. The final system allowed the group account managers to create or delete accounts for their own organizations without having to bother us. The managers could even change the passwords for people who had forgotten their passwords but only for people who were in each manager's particular group.

A.4.4.3 Lesson: Build systems that are extensible.

I wrote all of the software in the Perl 5 programming language. Perl is a great language for building large applications relatively quickly, and it runs reasonably fast. For a customer database, I used the Unix filesystem. A large directory called /vni/accts has a subdirectory for each user on our system. Inside each user's directory is a series of files, each file containing a different piece of information about that user's account. So the account type for Eric's account was kept in a file called /vni/accts/ericx/account-type, whereas the name of the reseller that owned Bill's account was kept in a file called /vni/accts/bill/owner.

As the system evolved, we developed three kinds of Perl programs. The first was a set of library routines that were used by all of the systems. These library routines managed the account database and the billing system. The second was a set of CGI programs that could be used to perform routine chores, like looking at a user's bill or adding an account. The third was a set of Perl programs meant to be run from the command line. These were administrative programs meant to be run by Eric or me.

A.4.4.4 Lesson: Automate everything you can.

In writing these programs, I had a simple rule. The first time I had to do a task, I would do it manually, simply to be sure that I understood what was wanted. The second time I had to do something, I would do it manually again, to be sure that I was doing it correctly. The third time, I would write a program.

Following this strategy, we soon ended up with dozens of small but useful programs. We didn't forget about them; they were all referenced on a web page. We set up the Vineyard.NET web server so that users, resellers, and administrators would all use the same URL to access the system: http://www.vineyard.net/start. The system automatically looked at the username of the person accessing the web page and made the appropriate commands available.

For the first few months, I had a single regret about the design of the system: I wished that instead of using the Unix filesystem, I had used an actual relational database, or at least a Perl DBM file. But as time went on I realized that the decision to use the Unix filesystem as our database was a pretty good one after all. Certainly the filesystem could handle it: I've been on Unix systems with thousands of users, and they store all of the user home directories in a single directory. Furthermore, using the filesystem meant that we could use all of the standard Unix tools, such as grep and find, to manage our user accounts database. I figured that when we hit 10,000 customers, we would probably have to do something else. But quite possibly, all we would have to do would be to add a second layer of indirection, changing Eric's directory from /vni/accts/ericx/account-type to /vni/accts/e/ericx/account-type.

A.4.4.5 Lesson: Don't reinvent the wheel unless you can build a better wheel.

As it turns out, I was wrong about the scalability of our file-based database: it started to get really slow not at 10,000 customers, but at 600. One of the reasons is the way that the Unix filesystem implements locking on directory inodes. Another problem, though, was that the file-based database had no indexing: searching for all of the accounts that were of type "dialup" or all of the customers for a given reseller required opening hundreds of files, which took a long time.

About three years into the project, it was clear that we needed to change to something new. My first impulse was to revamp my database using Perl DBM files. But my wife said that I would be far better off using a third-party database. After all, if Vineyard.NET ever hired somebody else, there was a remote chance that they might have knowledge about our third-party database, but there was no way whatsoever that they would know how to use "Simson's database." What's more, we might even find other people using the same third-party database, which would mean that there would be somebody we could ask for help.

I looked around and discovered that there were basically three free databases from which to choose: MySQL, mSQL, and PostgreSQL. Not really sure which was best, I wrote a common interface that would work with any of them (I thought that a Perl DBI was too complicated), downloaded them all, and tried them all out. mSQL was the fastest and seemed to be the most powerful, so we started with that one. After a few weeks, we ran into some significant problems with mSQL,, so I moved over to PostgreSQL. Although PostgreSQL had transactions, the lack of multithreading support proved to be a showstopper, so we finally ended up with MySQL. Of course, that was back in 1997. If we had to do it all over again, we might end up with a different database PostgreSQL now supports multithreading, for example.

A.4.5 Publicity and Privacy

With this basic business idea in place, we called up The Martha's Vineyard Times, one of the two newspapers on Martha's Vineyard, and asked them to write an article about us. The Times sent over a reporter with a camera, and a few days later the newspaper ran an article about Vineyard.NET with a photograph of Bill, Eric, and me.

The reporter wanted to print a phone number at the end of the article that people could call to get signed up for our service. This free advertising was precisely the reason that we had called the newspaper in the first place!

A.4.5.1 Lesson: Always be friendly to the press.

Unfortunately, our phone numbers were in a shambles. It was clear that we didn't want to use the number 696-6653 as our office number. First, it was too difficult to remember. Second, it was clear that we wanted as many of the 696-665x numbers as possible to be in our hunt group.

Eventually we picked the number 696-6688 as our office number. But that number wouldn't be installed for more than a week. In the meantime, the company was using my home phone number, which is the number that the newspaper ended up printing.

A.4.5.2 Lesson: Never give out your home phone number.

Letting the newspaper publish my home phone number was one of the biggest mistakes that I made throughout the entire Vineyard.NET project. For the next year, I received telephone calls on almost a daily basis from people who wanted to find out more about the Internet. It's true, the phone number was only printed once. But people clipped the article. People photocopied the article for friends. They wrote down the phone number, thinking that they would get around to calling it someday. I got those calls during the day, over the weekends, and even in the middle of the night (people who couldn't get to sleep, and who thought that they would be leaving a message on our answering machine).

It turns out that there were many other places that my home phone number had been used as well. My phone number was in the router MOTD (message of the day), because the MOTD had been written before Vineyard.NET became a commercial service, and it had never been changed. NYNEX had my home phone number listed as the official contact number for Vineyard.NET's account, because that was the phone number that I had asked them to call when the Centrex line was first set up. Months later, when Vineyard.NET started billing people's credit cards, my phone number was the phone number that was printed on our customer's bills because our bank had simply copied down my phone number from their records.

A.4.5.3 Lesson: It is very difficult to change a phone number. So pick your company's phone number early and use it consistently.

Perhaps I should have changed my home phone number, but I didn't. Still, it was hard not to get angry the thirtieth or fortieth time I got a phone call from somebody wanting information about Vineyard.NET. Indeed, sometimes I did get angry such as when I got calls at 8 a.m. on a Sunday morning. Unfortunately, every time I got angry at somebody for calling my home phone number, I was hurting the company, because the person at the other end of the line genuinely thought that they were calling the place of business, not my house.