This article is the fourth in a series that is designed to help readers to assess the risk that their Internet-connected systems are exposed to. In the first installment, we established the reasons for doing a technical risk assessment. In the second article, we started to discuss the methodology that we follow in performing this kind of assessment. The third part discussed methodology in more detail, focussing on visibility and vulnerability scanning. This installment will discuss a relatively unexplored aspect of Internet security, custom Web applications.

Web Application Analysis

Of all the possible services on the Internet DNS, e-mail and the World Wide Web are by far the most pervasive. (In fact, in July 2002, the monthly Netcraft Web site survey reported that 37,235,470 active Web servers were connected to the Internet.) Of these, Web services are the most complex and the most frequently abused.

Originally a simple text-based information service, the Web has developed into a highly functional interactive application development platform that is being used for almost every possible application on both the Internet and Intranet. Major vendors have recognized the power of the Web and have made significant investments in Web development platforms. These include Sun (Java), Microsoft (Active Server Pages, Site Server and Commerce Server), the open source community (PHP) and others (ColdFusion). Such platforms make it very easy for standard administrators to develop complex applications. A simple wave of the Web development wand and suddenly everybody's a programmer, even me! Web applications are often developed by under-qualified, inexperienced developers. Unknowingly many programmers make errors that provide potential intruders with precisely the vector they need to penetrate the private network. As these applications are built on standard platforms, they tend to look similar, and tend to have similar security weaknesses.

As an increasingly prominent technology on the Internet, custom Web applications must be considered part of every security assessment. Here's the problem though: we know that almost everyone who runs DNS is running BIND (the Berkley Internet Name Domain). Thus, if there is a vulnerability associated with BIND, then it will affect everyone. The vulnerability will become publicly known and the signature will eventually find its way into vulnerability scanners and vulnerability databases.

We can test systems using such scanners and identify vulnerabilities that may affect us. But how does a vulnerability associated with Joe Soap Inc's homemade customer support Web site become known so that it can be included in our handy vulnerability scanner? Answer: it doesn't. Because Web applications are so often "homemade", the vulnerabilities within them are unlikely to become widely known, so each application has to be looked at individually in an effort to identify security errors that the programmer may have made.

Web application assessments are a whole new ball game and present a whole bunch of new challenges for security analysts. Previously valid assumptions, methodologies, and tools are no longer applicable and even leading practitioners are still only beginning to understand all the possible issues with these kinds of systems.

One of the leading thinkers in this area is the OWASP group - the Open Web Application Security Project. The first draft release of their assessment framework is expected in mid-October of 2002. Another valuable resource is Web Hacking - Attacks and Defense by McClure, Shah and Shah. Also coming soon is a Syngress publication entitled Special Ops that will have a nice introduction to the topic. As always there are countless others.

Web Application Security Risk Testing

Our Web application testing is mainly done "black-box", which means that we examine the applications over the network without access to the source code. Of course, without access to the source code there's a fairly good chance that testers will miss something important. Call this the "fuzz factor". We've addressed the "fuzziness" of Web application testing by introducing two frameworks of our own. The first is a simple list of questions that need to be asked when considering the application and the second is a simple list of common programming errors that should always be checked for.

Even though every programmer (and thus every application) is unique, one does see the same errors being made consistently. We've compiled a list of common programming errors that can be checked for, even over the network. I'll describe the frameworks in some detail a little later, but first I'll need to quickly cover some basic World Wide Web concepts that readers will need to understand.

• HTML stands for Hyper Text Markup Language. It's the simple language that is used to describe how Web pages should be formatted and displayed. HTML uses commands called "tags" to tell the browser how a Web page should look and behave. Examples of tags include <A HREF>, which indicates that the text to follow is actually a hyperlink to another page or <B>, which indicates that the text to follow should be displayed in bold characters.
• HTTP is the Hyper Text Transfer Protocol. It uses a TCP connection (typically on port 80) to transfer data between the Web server and the browser. HTTP is used to transport all the files used in HTML, including text, graphics, sound, video and active elements like Shockwave, Flash and Java Script.
• A form is a special kind of HTML element that allows the user to submit data back to the Web site. A form consists of any number of fields of various types. A hidden field contains a value that is not displayed by the browser (more about this in the fifth installment of this series) and is occasionally used by programmers to submit control information that the user doesn't need to know about. When dealing with normal form input fields, the user can enter data into the field, which is then passed back to the Web site and extracted and processed by a server-side application, typically a CGI (common gateway interface, see below for more detailed explanation).
• CGI is the Common Gateway Interface. It describes a standard way for Web servers to pass information to programs and send the output from those programs back to the browser to be displayed. With CGI, a Web developer is no longer restricted to using only static pages. Instead, one can accept input from the user, pass that input to a program to be processed, and then send the output to the browser to be displayed. CGI is an old technology but has recently really come to its own. Other technologies like Microsoft's ASP (Active Server Pages) perform a similar function.
• A Web site that wishes to keep track of its visitors will often make use of cookies, small data objects (usually stored as files) that are sent to the browser to be stored for later use. The next time the user visits that site the cookie is automatically resubmitted, complete with all the information the Web site previously wrote there. You may have seen this mechanism in use by sites that "remember" your preferences from one visit to the next. Actually, your browser is using cookies to remind them.
• A script is a small program that is not compiled into executable format before it is run. Instead, each line is read, compiled and executed by an interpreter whenever the program is run. Scripts are powerful because they are lightweight and easy to write and can run on multiple platforms without being modified. Increasingly Web applications are being written in scripting languages like ASP, PHP, PERL and Java.
• SSL is the Secure Sockets Layer. It's a protocol that runs on top of TCP and uses a combination of public key and symmetric encryption technology to ensure confidentiality, integrity and authenticity for HTTP. When you select a Web site with a URL that commences with the letters "HTTPS://" you're instructing your browser to tunnel the HTTP communication over SSL. SSL is a very powerful mechanism, but is most often used only to authenticate the Web server and to encrypt the link between the browser and the server.
• ODBC is the Open Data Base Connectivity protocol. It's a standard way for applications to communicate with databases without needing to understand the peculiarities of that particular database implementation. ODBC is typically used between the Web server and the database in architectures like the one we depict in Figure 1 below.
• Application development environments are used to make the development of complex Web applications quicker and easier. This will typically involve a number of elements, including: a Web-optimized scripting language, reusable components, samples and example scripts, session key generation, database connectivity, and the like.

Irrespective of the development platform being used, the vast majority of Web applications have the same basic architecture, as depicted below:

Figure 1: Typical Web application architecture

This picture depicts a popular network architecture for Web applications. The Web server is located in a specially protected network segment called a Demilitarized Zone (DMZ). Two firewalls are used: one to protect the DMZ from the Internet and one to protect the internal LAN from the DMZ. This is sound security thinking and is done to minimize the impact if a Web server is compromised. However, two channels must be opened for this architecture to work. The user must be able to connect to the Web server from the Internet and the Web server must be able to connect to the database server. Remember this configuration, as these same two channels are what an attacker will exploit to gain access to the internal network.

Web Applications: Key Questions

Now we have the background we need to understand Web application assessments. We've also looked briefly at a fairly simple, although very common network architecture. So, let's take a deeper look at the first of the frameworks we mentioned earlier. What questions should you ask yourself when faced with a custom-built Web application? These are generic questions that border on the philosophical. However, they are important if we hope to see beyond the surface into the heart of the application we are assessing.

The following are the questions we need to ask:

Question 1: Who are these people?

The design and architecture will more often than not reflect the style and culture of the organization that wrote it. Conversely, much can be learned about the design and architecture of an application by understanding the style and culture of the organization. We seek this understanding at two levels:

• Generally: What is the group we are dealing with? What is their core business? What is their style and culture?


• Specifically: Who designed and developed the application we are considering? What is their background, their training and their style?

The answers to these questions will give us insight into how the application is likely to be constructed and where possible errors are likely to be found.

Question 2: How does this thing work?

Next, we need to understand how the application in question was put together. The key to success here is an enquiring mind. No stone should be left unturned in your effort to understand what makes the application tick. Here are some specific elements you should be considering:

1. Is there interactivity? If so, where is it? Not every site is interactive. And some sites are more interactive than others. You need to identify the elements of the site that are interactive and determine how interactive they really are.


2. How did they build this system? Remember the application development platforms that we discussed earlier? Each development framework has its own inherent strengths and weaknesses. By identifying and understanding the base component on which the application was built we know where to start looking for possible security vulnerabilities.


3. Where do they get their data? An interactive application needs to get its information from somewhere. As Web applications are stateless, a place must be found to store the user's data. Determining where the back-end data source is located will teach us a lot about how the application works and how it might be compromised.


4. How do they get their data? So the site is getting its data from somewhere else. Perhaps even a remote server. How? There are many options, including HTTP requests, XML, basic file access and the ever-popular ODBC. Again, each of these approaches has strengths and possibly weaknesses that we need to identify.


5. How do they know who I am? Many sites will allow you to work anonymously; that is, without first authenticating you. However, many sites will want to know who you are before they grant you access. If this is the case you need to understand how authentication is performed. What is the method (e.g. Basic Authentication) and where is the user data stored (for example, in the NT SAM).


6. How do they keep track of state? Once a user has been authenticated, a stateless application needs to keep track of a person as she/he navigates their way around. State tracking is also needed for complex processes, where the output from one step needs to be remembered as the input to the next. In each case, state tracking is an area much prone to error and one must have an in-depth understanding of this when an application is assessed.

Question 3: Why did they do it this way?

Whatever the answers to the previous questions may be, and whatever the architecture that you've mapped, there is always a reason the system was built in the way it was. If we can understand the motivating factors behind the design, we can gain valuable insight into where the system might be vulnerable. Here are some common motivating factors that we'll be looking for:

• Laziness: Often a specific architecture is chosen simply because it's the cheapest or the easiest. A lazy design is an insecure design.


• Policy: Often technical people are forced to stick with certain technologies because of high-level strategic or political decisions within the organization. For example, one often sees an organization strategically aligned with Microsoft or with open source technologies. Such architectures tend to be standard and are well known.


• Inexperience: We've already alluded to the fact that many Web application developers are inexperienced and under-skilled. Such people tend to "borrow" and copy code from example sites and other public sources and are prone to making silly, amateur mistakes.


• Over-experience: In our industry people who are highly skilled in one area often achieve this at the expense of other areas. Identifying the developer's particular "hobby-horse" will often tell you where not to look for vulnerabilities and, by extension, where some weaknesses may lie.

Question 4: What are typical mistakes in this architecture?

Now that you've "scoped" out the system you're assessing, you can begin to look for security errors that might be expected. You're working your way under the skin of the developer and attempting to see the mistakes that he/she may have overlooked. In the sections that follow (in the fifth and final part of this series), we will be discussing some common categories of programming errors and looking at some specific examples. Now that we have laid the foundation for understanding what Web applications are and how they work, we can explore how to assess them for Internet security risks, which we will do in the next (and final) installment in this series.

Charl van der Walt works for a South African company called SensePost that specializes in the provision internationally of information security services, including assessments of the kind discussed in this series. His background is in Computer Science, he is a qualified BS7799 Lead Auditor and he has been doing this kind of work for about five years now. he has a dog called Fish.

To read the next installment of this series, click here.