One of our customers had a unique challenge of moving web servers. Their site was huge, with one directory having over 200GB of images. They opted to do a partial migration, copying over the website as is first before the final switch over.
The final switch over required to copy over only the latest files created or modified after a particular date.
The most efficient way would have been the rsync utility. Unfortunately, this was not an option as we did not have SSH access on the new service, so we had to find an alternate way.
The objective was simple;
- Find the files
- Archive/compress them
There are two commands that required to be run, the first one was to search for new or modified files after a particular date and the second one was to create a tar file.
The dry run command looked like so:
find /path/to/folder -type f -newermt '2017-04-01T00:00:00' -print0
Let’s break this down:
- find /path/to/folder
- This defines where to search
- -type f
- We’ll be looking for files only, recursively.
- -newermt ‘2017-04-01T00:00:00’
- The date from where we want to search from
- This outputs the files so it can be piped into the tar command
For the second objective, we piped in the tar command to accept the output from the first. This would be appended to the original command.
- | tar -czvf /backup/archive-name.tar -T –
- Begin piping into the tar
- The tar will compress and output the progress (verbose)
- -T – takes in the files to archive from the previous output
Here is the final command:
find /path/to/folder -type f -newermt '2017-04-01T00:00:00' -print0 | tar -czvf /backup/archive-name.tar -T -
I hope this will be useful for anyone with a similar requirement.
If you’ve stumbled across this post, then you’ve recently become a victim of the most famous way of uploading a malicious script to a website – PHP file uploads.
Many websites and web applications that are developed using PHP & MySQL allow users to upload files, photos and documents to the server. Normally, the upload script receives the files and moves (or writes) it to a directory (folder) with write permissions. If you are on Linux, then this would mean that your folder CHMOD value is 0777.
The changing of CHMOD value to 0777 practically allows anyone in the WORLD to write to your folder and is therefore not recommended. However, many a times we are left with no option but to do so – depending on the servers’ environment.
How can this be a problem?
This allows attackers to upload upload a malicious PHP script to your directory, which they will then execute by accessing it. This script could either be a mass-mailing script or a malicious script to gain access to your account (or web server).
It is said that prevention is better than cure – and therefore, it is important to prevent these scripts from being executed by the server. This can very simply be done by adding a few lines of code to your .htaccess file. If your directory is supposed to hold photo files only, then the following code is recommended to prevent scripts from being executed.
It is also important to disable directory listing on these folders.
# Disable Directory Listing
Options All -Indexes
# Allow access to these file extensions only
Deny from all
The above will add an extra layer of security to your web application.
Using the following command in SSH, you will be able to extract files from your tar file.
tar -xf file.tar
Use the following command if you want to create a tar and compress a file (or folder).
tar -czf /path/to/save/file.tar folder_or_file_name
The options used in the above command are outlined below for your understanding.
-c = create
-z = compress a file
-f = use the file mentioned instead of tape drive
If you have a dedicated server or a VPS – or any Linux machine with SSH access, the following commands may be useful to you if you are looking to find some files quickly. PuTTY is a good telnet/SSH client if you don’t already have one. Let’s get started.
#1: If you are looking for a particular file in /home directory
find /home -name filename
#2: If you are looking for files whose name is starting with “filename” in public_html directories only, you can use the following command:-
find /home/*/public_html -name filename*
#3: Instead of viewing your search results in command line, you can write it to file by using this:
find /home/*/public_html -name filename* > /home/filename_search.txt
#4: Alternatively, you can send it via e-mail directly from server using the command below. Don’t forget to replace “Subject Here” with your email subject, and “[email protected]” with your email ID.
find /home/*/public_html -name filename* |mail -s "Subject Here" [email protected]
I hope this was useful.
Earlier this week, I was transferring all my important data from my external 120GB USB hard disk to my new laptop. During the process, the transfer became unbearably slow. I decided to cancel it and reboot my computer to get a fresh start.
To my dismay, Windows Vista detects the hard drive but is unable to read it prompting me to format the drive. After a bit of reading, and trying to access the drive in different ways – I got the following error message.
Disk is unaccessible.
The file or directory is corrupted and unreadable.
It was impossible for me to let go of all my data and thus, I started looking into ways to recover it.
During the process, I realized that the format of my drive had changed from NTFS to RAW – which was rather strange. Searching through Google on Data Recovery Software, I stumbled upon a neat software called R-Studio. I downloaded its trial version to give it a try.
It took approximately 7 hours for it to scan through my corrupt hard drive – after which I could view all my folder structure & files in the hard drive. You can only preview data or recover upto 64 Kb in the demo version of R-Studio. I tested its recover data tool and was very impressed to see that it managed to recover my data.
I purchased the full version of the software for a worthy price of USD 79.99 – got the license key immediately after making payment and started off recovering my data.
Thanks to R-Studio, I have recovered most of my data – besides some that resided on bad clusters.