Apache has default page to tell us that the installation was succesful. Once, we are satisfy with the result, remove the redirection by doing this:
debian:~# cd /etc/apache2/sites-available/
debian:/etc/apache2/sites-available# vi default
find the RedirectMatch line, in my case it is in line 46. Comment it out by placing '#' in front of it:
#RedirectMatch ^/$ /apache2-default/
Finally, check whether the redirection is removed by creating a simple index.php, don't forget to restart the apache, then go to your web browser to check whether you are redirected or not.
debian:/etc/apache2/sites-available# /etc/init.d/apache2 restart
Forcing reload of web server (apache2)....
You can simply remove the apache-default directory as it is unused anymore.
debian:/var/www# rm -rf apache2-default/
Now, your web server ready to serve!