The purpose of this FAQ is to gather together some of the common questions asked when people develop Web Applications. Although there are FAQs for the individual components, (HTML, Javascript etc) often the answers encompass more than just a single language. This FAQ is designed to fill that hole.
How can I send an Email from a Web Page and does the user have to know
There are two ways in which an email can be sent from a Web page. The first is that the CGI that processes actually sends the email, rather than from the browser.
The second method is that the action specified on the form consists of the mailto tag e.g.:
<FORM ACTION="mailto:fred@bedrock.co.uk">The contents of the form will then be mailed to the named account with the contents presented in the standard CGI style of name value pairs divided by &s.
There are a couple of twists involved when a mailto tag is used as an action. Firstly the user will always be notified that an email is to be sent so that they can prevent their email address being returned to the web site.
The less than scrupulous may think that they can overcome this by using javascript to do the submission when the page is first loaded by trapping the onLoad event. However this does not work in Internet Explorer as the confirmation dialog box is still displayed. Netscape on the other hand go one better in that the submit() function will not work if the action starts with a ‘mailto’.
The second problem is that, unlike a normal submission, the page is not replaced when a mailto is used. Consequently the user requires two actions to send the mail and move on to the next page.
Of course the solution would be to do the submission from javascript and then load the new page. However as we have already said this is a problem particularly with Netscape.
There is a method to overcome this although it is not ideal. The following code demonstrates this.
function redirect()
{
document.location="thankyou.html";
}
function sendmail()
{
if(navigator.userAgent.indexOf('MSIE')<0)
{
setTimeout("redirect();","4000");
return true;
}
else
{
document.forms[0].submit();
redirect();
return false;
}
}
<FORM ACTION="mailto:fred@bedrock.co.uk" onSubmit="return sendmail();">
<TEXTAREA NAME="comment"></TEXTAREA>
<INPUT TYPE="SUBMIT">
The key to this is that if the browser in use is Internet Explorer then the submit() method will be used. If any other browser is in use then a timer will be set up and the normal submission process allowed to continue. Hopefully this timer will allow the user enough time to see and press the confirmation button that allows the email to be sent before the redirect function is called to change the page that is being displayed.
People often asked if there is any way around this so that they can illicitly send an Email without the users knowledge. The answer is simply no.
How do I have two submit buttons in one form
Often within HTML forms we will want to have some information displayed but let the user do two or more different things with it. The obvious thing to do is to provide multiple submit buttons to allow for these multiple actions.
The problem that then arises is how does our CGI know which one was pressed.
A simple rule of Forms/CGI that is often not mentioned in books on this subject is; a name value pair will be returned for a field only if it has both a name and a value.
This applies to all field types including submit buttons but with the caveat that only the name value pair of the submit button pressed will be returned. So if we used the following HTML
<FORM NAME="myform" ACTION="mycgi.cgi" METHOD="POST"> <INPUT TYPE="TEXT" NAME="username" VALUE="FRED"> <INPUT TYPE="HIDDEN" NAME="userid" VALUE="123"> <INPUT TYPE="SUBMIT" NAME="isubmit" VALUE="Delete"> <INPUT TYPE="SUBMIT" NAME="isubmit" VALUE="Update"> </FORM>
If the user presses the delete button then the name value pairs returned would be
Username=FRED Userid=123 Isubmit=Delete
Has they pressed the update button then the same fields would have been returned but the value of the isubmit field would now be "Update".
Consequently with the CGI we can test this value to determine our next action.
Now sometimes one doesn’t want a CGI to be able to do both things as it may complicate and already complex task but we still want the ability to have two buttons in the form.
There are two potential solutions to this. The first is that the CGI that is called in fact just determines the action to be taken and calls the appropriate program. This is potentially expensive at the server end of things but will work under all circumstances. The second solution, which involves javascript, won’t work if the browser has this feature turned off.
We’ll use the same HTML as before but note that some javascript has been added.
<FORM NAME="myform" ACTION="javascript:void()" METHOD="POST"> <INPUT TYPE="TEXT" NAME="username" VALUE="FRED"> <INPUT TYPE="HIDDEN" NAME="userid" VALUE="123"> <INPUT TYPE="SUBMIT" NAME="isubmit" VALUE="Delete" onClick="document.myform.action=’delete.cgi’;return true" <INPUT TYPE="SUBMIT" NAME="isubmit" VALUE="Update" onClick=document.myform.action=’update.cgi’;return true"> </FORM>
The way this works is firstly the Action has been replace with javascript:void() which is a place holder. If javascript is turned off nothing will happen when the user hits the submit button as no CGI has actually been specified.
The other change is the to bits of javascript that have been inserted into the onClick clause. When the button is clicked this code will be executed and will cause the action to be set to the appropriate CGI. They then return true which means that the process of the submission can be completed thus the correct CGI is called.
You will note that the name is still present on the submit button and thus a value will still be passed back to the CGI so it is possible to combine the two techniques. Thus several actions could be handled by one CGI (using the name to determine the action) whilst other actions are handled by a different CGI.
The concept of the name value pair being returned for a submit button has one variation which is when an image is used for a submit button. E.g.
<INPUT TYPE="IMAGE" SRC="button.gif" NAME="isubmit">
In this case two name value pairs will be returned to the CGI namely isubmit.x and isubmit.y the values associated with these two variables will be the x and y co-ordinates of the point at which the button was clicked by the mouse.
Generally the existence of the name/value pairs is used rather than the actual contents although one could use a single image that contained a number of ‘buttons’ and decode the value to determine the actual action.
The bigger problem that this presents is that if multiple image buttons are used then you can’t use the value to determine which one was pressed. Even if a value attribute were contained with the HTML this value wouldn’t be returned.
So in this case each button would need to be named differently in the following fashion.
<INPUT TYPE="IMAGE" SRC="button1.gif" NAME="button1"> <INPUT TYPE="IMAGE" SRC="button2.gif" NAME="button2">
How do I make my site secure without authentication turned on
It must firstly be stressed that whilst there are a number of techniques to implement secure sites without authentication turned on these techniques do not offer the same level of protection that the authentication schemes provide.
In order to provide our own authentication system we will need to mimic the standard method to a certain extent and to that end before describing how we can implement it is worthwhile examining how standard authentication works first.
When authentication on a site is turned on and a browser makes a request of a page from it for the first time the actual request is rejected with a 401 status HTTP message. This message means that a connection is refused unless a user name and password are received. In addition it informs the browser of the authentication methods acceptable and the realm for which the request is made.
Typical authentication methods are Basic, Digest and NTLM (on Microsoft Servers only).
The realm value varies from one server to the next. In the original implementation of the approach the realm was just a name which had no specific meaning but provided a grouping mechanism for users. On Microsoft’s IIS the realm is the name of the Server Machine.
Upon receipt of the message the browser pops up the username/password dialog box. Once completed it re-requests the page but this time transmits the username and password. Assuming that these are correct the Web Server will return the requested page. Now whilst the browser continues to make further requests within this realm the username and password will be transmitted with each request.
When we come to mimic authentication it is important to realise that we will need to server all pages at least and possibly all resources from a CGI that can undertake the authentication. Otherwise all we will be doing is protecting a single page and not all the pages as the genuine schema would.
Rather than rely on the dialog box within the browser we must create a complete page that contains a form with fields for the username and password. (Optionally a small window could be popped up using javascript thus making it appear closer to the normal dialog box).
This form will be processed by a CGI that will, as we will see, be used to serve all resources, that requiring securing. This CGI should validate the username and password in anyway that it sees fit but from this derive a unique ID. This may be a static value such as an ID from a user record in a database or a calculated non-reoccurring value such as the number of seconds in the current day.
This value will then be used in one of two ways. Either the CGI can send a cookie back to the browser containing it or it can be used as a hidden field within a form or additional parameter on a URL. Either way the CGI should also display the next page.
To secure other resources and pages it is important that they are all accessed via this same CGI or a related CGI that checks that the ID is present before processing the request. Typically if static pages are being used then the CGI will need to server these an will need to know what page they must serve. If the embedded ID approach is used as opposed to the cookie method then the CGI will actually need to parse the page so that the ID can be inserted into the value clause of hidden fields or within the URLs of links. This approach is somewhat complex in standalone CGIs but in systems such as ASP it is relatively simple to do.
If the CGI is serving up static pages it is important that the directory that it resides in is not within the structure accessible by the web server as this would mean that if a user entered the URL correctly they would have bypassed the CGI.
There are three additional features that can be built into this approach to add a bit more security.
Firstly when the login page is processed a check can be made that he Request Method was a POST and that the page that the form was submitted from was the one that we expected. The referring page can be determined by looking at the HTTP_REFERER environment variable which should contain the complete URL of the logon page. This approach will ensure that it is not possible to login via a browser without displaying the correct login screen first. This does not prevent someone using telnet to overcome this however.
The second ‘trick’ is to use a revolving ID or an ID pair. The technique with a revolving ID is that one each page output the ID will change. At the server end a record will be kept of what the last ID this is checked when a request is received where upon a new ID is issued. The method of calculating the ID should be such that it is non-predictable. This could be a number of things; the number of second since a predetermined time less the process id of the Web Server for example. This approach makes it difficult for someone to ascertain an ID value used and then re-use it at a later date.
The ID pair is similar in that they will change each time in a similar manner but two IDs are issued each derived using a different method. The Ids are stored and checked as described before. Using two Ids just makes the task of attempting to predict the ID that much harder.
The third and final trick is that in creating the ID and element of the calculation involves a reference to the next file to be retrieved. Whatever algorithm is used it is important that the file reference can be deduce from the ID once it is returned. In making the deduction we know what file to output but we also know that an attempt to predict the ID will only give access to one file and not any file.
How can I track who is logged on to my server
Strictly speaking the answer to this question is that you can’t. To understand this it must be appreciated that a Web Session actually only lasts for the period between a web page being requested and it being returned. When a second page is requested this constitutes a different session (or transaction to give it its proper name).
Within the HTTP protocol that governs the Web there is no real concept of logging in or logging out so you can in theory only track a user as being logged in whilst the page request is being processed by the server. After that you don’t know if the are still looking at your web site, connected to the Internet or even have their PC turned on.
Now some people have spotted that there web server supports Keep Alive connections an believe that this may help them. Unfortunately this is not the case as Keep Alive connections refer to the TCP/IP connection (a layer below HTTP and not accessible from it as such) . In fact whilst keep alives may be used they are advisory in that either the client or the server may decide not to keep the connection open. Further more even if the connection is kept open the user may still not request a further page.
However if you wish to monitor the number of open connections to you web server this can be achieved by using a TCP/IP tool such as netstat which will show all open connections.
If you are using a Web Server that supports a session concept such as the session within ASP pages then it is possible to track all open sessions by copying information from the session level to the application level. However the existence if the session does not indicate that a user is still ‘connected’.
The next issue in this area is how do you track that two separate page requests actually come from the same user.
If you have authentication then this can be determined by a CGI by looking at the REMOTE_HOST environment variable that will contain the username used to login. If on the otherhand authentication isn’t turned on then the only information that gives a clue to the users identity is in the environment variable LOCAL_ADDR. This contains the IP address to which the requested page should be returned. However this doesn’t always refer to a individual user. If an access is made via a proxy then the IP address seen will be that of the proxy not the user. If a number of people access your site through the same proxy then multiple pages may be sent back to the same IP address but this could refer to a number of different people.
Conversely many ISP utilise the Dynamic Connection Host Protocol (DCHP) which means that a user will have a different IP each time they log in. Thus even if no proxy is involved you cannot tell if two accesses from the same IP address refer to one person or two.
Probably the best answer to this question is ‘Why do you want to know’ because the chances are that you are trying to collect information that is not really valid in the context of the Web.
How can my CGI tell if the browser has javascript turned on
If you include validating features, for example, using javascript it is potentially a serious problem if the user disables it whilst once you page has loaded. So it can be important to detect that this has happened when we are processing a form within a CGI.
The key to detecting that javascript has been turned off is to rely on it failing by ensuring that a javascript task is executed immediately before the submission occurs.
The key to this technique is to include a hidden field within the form that contains the javascript status. Initially this is set to a value to indicate that javascript is turned off. The onSubmit event is then used to change this value to indicate that javascript is turned on. If in fact javascript is turned off the onSubmit event handler will never be called and the CGI will receive the appropriate value. The example code below demonstrates this.
<FORM METHOD="POST" ACTION="mycgi.cgi" onSubmit="this.javaflag=’on’"> <INPUT TYPE="HIDDEN" NAME="javaflag" VALUE="off"> <INPUT TYPE="TEXT" NAME="ifield"> <INPUT TYPE="SUBMIT"> </FORM>
How does client pull work
Client pull utilises the ability of HTTP to redirect a browser to a different page combined with the ability of the Web Browser to to set up timed events. To undertake a client pull the following line should be inserted in the head section of the HTML
<META HTTP-EQUIV="Refresh" CONTENT="10; URL=www.mydomain.com">
The way that this tag works is that the value 10 is the number of seconds that the browser must wait until it loads the page specified as the URL.
It has been noticed that the time specified sometimes is longer than actually specified although the actual cause of this is not known.
If a time of zero seconds is specified the browser will attempt to load the new page as soon as the tag is processed.
All the rules associated with the URL and with caching are the same as if the page was loaded in a more conventional manner.
Those who understand how the META tags operate will know that a web server parses the head section of an HTML page and converts the META tags to HTTP headers. However this is not true with this is not true of this particular Meta tag which is wholly processed by the browser.
How do I stop the back/Reload button working
This is a very frequently asked question to which strictly speaking the answer is that you can’t prevent either of these buttons from working.
However what you can do is to minimise the effect or problems that this button can cause you. Before explaining how to deal with this situation lets first look at a typical problem that can be caused by their use.
Imagine a page that contains a list of records each within a form which allows the data to be modified and, via the submit button, a database updated. In addition there is a form that will allow the user to add a new record. When we do an update or an add we update the database and redisplay the page.
However if the user hits the reload button they will normally be asked if they want to repost the data. If they select yes then the original call that re-displayed this page will be made to the CGI again. If this was for an addition a second record will be written to the database unintentionally.
The trick to preventing such problems is that after we update the database we do not redisplay the page but redirect the browser to another page. This second page is the one that then displays the data. As a consequence if the user presses the reload button the page they were redirected to is loaded rather than the data being re-posted. If the back button is pressed then the redirection is loaded which will then immediately redirect them again to the page that is currently loaded.
How you do a redirection varies slightly depending on whether a CGI is involved, an ASP page or a page of HTML.
If plain HTML pages are involved (that’s static HTML files not HTML generated by a CGI) the following line within the head section effects a redirection.
<META HTTP-EQUIV="Refresh" CONTENT="0; URL=www.mydomain.com">
If a CGI is involved the following line should be output to the client (this can also be preceded by cookie settings but nothing else whatsoever)
Location: http://www.mydomain.com
This should be followed by two carriage return line feed pairs. Also note that the full and absolute URL must be specified, relative URLs are not supported.
If an ASP page is involved the following code
Response.redirect("www.mydomain.com")
implements a redirection.
Many people see the mechanics of this approach causing them to duplicate the code of the first page for the final page but you do not generally need to do this. For example the redirected URL could be modified so that it refers to the initial page but with an additional parameter to indicate that no update is required. Alternative the code that does the update and the code that displays the page can be split into two separate files/CGIs.
The only real problem that this approach does not address is that if javascript variables are being maintained within a page and one is try to prevent the page from being reloaded so that they are not lost. The solution to this is fairly simple in respect of the back button in that all the variables should be held within the frameset rather than a frame (even if there is only one frame) thus when the back is pressed the frame is changed but not the frameset and the variables. However if a reload is done then they will still be lost and short of storing them as Cookies instead there is no solution to this problem.
How do I stop pages being cached
To stop a page from being cached HTTP provides a special header which if output from a CGI should take the following form:
Pragma: No-cache
If a plain HTML page is used then a META tag should be included in the head section which should be as follows:
<META HTTP-EQUIV="Pragma" CONTENT="No-cache">
The issuing of this header should cause the resource not to be held in a proxy cache or a browser cache and is relevant to images, javascript files as well as HTML files (although it is only HTML that can use the META tag).
This header didn’t exist in version 0.9 of the HTTP protocol and as a result in order to facilitate the prevention of caching an expires header would be issued with a date that was already past. This worked because although the browser would store the resource in its cache when it next came to display it it would see that it had expired and request a fresh copy.
As a result of this little bit of history some people have used both the no-cache header and the expires header so that they can ensure that the page is not cached with either a modern browser or a very old browser.
Frankly this is unnecessary these days as 0.9 browsers are extremely rare. But more importantly the two headers are mutually exclusive in that one is meant not to cache at all whilst the other is meant to cache until a given date. From the browsers point of view it has two contradictory headers and it must ignore one of them. Unfortunately the one that will be ignored will vary from browser to browser as a result one can find that the page gets cached, even though this cached version never gets reused.
Despite the rarity of such browsers the idea of outputting both headers seems to have passed into Web folklore and is frequently still quoted as the way of preventing pages from caching.
Earlier it was mentioned that the no-cache header could be used on all resources including images. However there is known bug in some versions of Netscape in that images that are marked as no-cached will eventually be cached after they have been referenced a few times. It is well to be aware of this bug, for which there is no workaround, as many people who have come across it unwittingly believe there code is at fault rather than the browser.
Is it possible to automatically submit a form without the user doing anything
Within javascript this is fairly simple in that the form has a method called submit(). Calling this function is almost identical to a user pressing a submit button.
There are three differences between calling this function and pressing the button.
Firstly if the submit button has a name e.g.
<INPUT TYPE="SUBMIT" NAME="mysub" VALUE="Update">
When clicked the name value pair of mysub=Update will be returned to the CGI. If the submit function is called then no such name value pair will be returned.
Secondly any code associated with the onSubmit event handler will not be invoked. However you can invoke this by calling onsubmit() as a function.
Lastly on Netscape only if the action of the form contains a mailto tag the submit is disabled completely. On Internet Explorer it will work, although the pop up box informing the user that there email address is going to be transmitted will also be displayed.
How can I make a field read-only in all browsers
Microsoft’s Internet Explorer supports a read-only attribute on input fields that means that the data in the field is displayed but the user can’t edit it. Unfortunately Netscape and virtually all other browsers don’t support this attribute however there is a way to achieve this using Javascript.
The key to this technique is to shift focus away from the field as soon as the user set focus to it. We do this by intercepting the onFocus event as follows
<INPUT TYPE="TEXT" NAME="field1" VALUE="123" onFocus="this.blur();">
The blur function sets the focus to some other field or component as soon as this field is selected either with the mouse or the via the keyboard. What gains the focus afterwards is difficult to determine because IE and Netscape operate in different ways in this respect.
Is it possible to manipulate a page in a different frame that comes from a different server
There is a security hobble within web browsers that prevents javascript in one frame from manipulating another frame if they do not both come from the same web server. This hobble is there to prevent a ‘bogus’ site from stealing data from a trusted site without the users knowledge. However there are occasions when this form of manipulation is required for honest reasons.
The key to overcoming the hobble is to fool the browser into thinking that all frames come from the same server. Take the following example frameset definition
<FRAMESET COLS="10%,*"> <FRAME NAME="nav" SRC="http://mydomain.com/nav.html"> <FRAME NAME="main" SRC="http://www.yahoo.com"> </FRAMESET>
In this example any javascript in either this document or nav.html cannot manipulate the fields in the ‘main’ frame. Now look at the following example.
<FRAMESET COLS="10%,*"> <FRAME NAME="nav" SRC="http://mydomain.com/nav.html"> <FRAME NAME="main" SRC="http://mydomain.com/cgi/getit.cgi?url=www.yahoo.com"> </FRAMESET>
In this example you will notice that the source attribute of the second frame now points at the same domain as the first (ie mydomain.com) so any javascript problems are now resolved.
However it begs the question of what does getit.cgi actually do. The answer to this is quite simple although its implementation is less so. What the CGI must do is connect to the site, specified by the ‘url=’ value and fetch the page from this web server. It then needs to just output the data received. There is a slight wrinkle to this. If the web page contains any relative URLs then they will actually refer to mydomain.com rather than yahoo.com. This can be resolved by a small additional bit of processing by the CGI that inserts the following tag into the code:
<BASE HREF="http://www.yahoo.com">
This way any relative URLs within the document will be treated as relative to yahoo rather than our domain.
There is a potential snag to this approach in that if a base tag is already present then our base tag will be undoing its job. So we must ensure that we only include the base tag where one isn’t present or we must ensure that our base tag comes before the ‘included’ one.
If a URL is proceeded with the protocol specifier of ftp rather than http then the browser will connect to the ftp site of the specified domain. But it should be noted that this is the FTP server that is being used not the Web Server thus you will not actually be seeing the files that make up the web site (unless the FTP server has been made to point at them).
Such a URL can be entered within a link EG <A HREF="ftp://mydomain.com/"> or within the URL line of the browser.
There is one major restriction in that only anonymous ftp is supported. If a ftp server requires a username and password then it is not possible to surf is with a browser.