need help in my SH script please for Rsync

[collen]

Hi I need help with my SH files please. Its for Rsync command to backup my files to other vps but it seems its not working not really sure why.

First File.
GENKEYS.SH (run on main server)

#!/bin/bashcd /etc
chmod 777 *.sh
echo "Done: setting chmod 777 *.sh ..."
mkdir /root/.ssh
rm -f /root/.ssh/id*
rm -f /etc/destinationetc/.ssh/authorized*
rm -f /root/.ssh/known_hosts
echo "deleted existing keys...."
echo "SOURCESERVER will create the public/private key pair to be used for authentication."
echo " ***********************************************************************"
echo ""
echo "*Note: Do not enter a passphrase for this, just hit ENTER when prompted."
echo "*Note: Do not enter a passphrase for this, just hit ENTER when prompted."
echo "*Note: Do not enter a passphrase for this, just hit ENTER when prompted."
echo "*Note: Do not enter a passphrase for this, just hit ENTER when prompted."
echo "*Note: Do not enter a passphrase for this, just hit ENTER when prompted."
echo "*Note: Do not enter a passphrase for this, just hit ENTER when prompted."
echo ""
echo " ***********************************************************************"
ssh-keygen -t rsa
echo "DONE: SOURCESERVER created the public/private key pair @ /root/.ssh/id_rsa.pub and /root/.ssh/id_rsa.pub"
echo " ***********************************************************************"
rm -f /etc/destinationetc/.ssh/id_rsa.pub
rm -f /etc/destinationetc/.ssh/authorized_keys
echo ""
echo "deleted existing keys...."
echo ""
cp -r /root/.ssh/id_rsa.pub /etc/destinationetc/.ssh
cd /etc/destinationetc/.ssh
mv id_rsa.pub authorized_keys
echo "DONE: Generated fresh authorized_keys file to " 
echo "/etc/destinationetc/  ..."
echo ""
echo " ***********************************************************************"
echo ""
echo "DO NOT forget to edit the file "
echo " /etc/destinationetc/.ssh/authorized_keys  ..."
echo "at Main Server before deploying to other server....."
echo "...look for root@domain.com  and then edit it into"  
echo " root@64.xxx.xxx.81 format ..."
echo ""
echo ""
echo " ***********************************************************************"
echo ""
cat /etc/destinationetc/.ssh/authorized_keys
echo ""
echo " ***********************************************************************"
echo "Done.........."
echo "Done.........."

So what I do is run this file on the old or main server.

2nd File.
SETKEYS.SH (run on backup server)

#!/bin/bash
cd /etc
chmod 777 *.sh
echo "Done: setting chmod 777 *.sh ..."
mkdir /root/.ssh
echo "Done: mkdir /root/.ssh"
cp -r /etc/.ssh/authorized_keys /root/.ssh
echo "Done: cp -r /etc/.ssh/authorized_keys /root/.ssh"
cd /root
chmod 600 .ssh
cd .ssh
chmod 700 authorized_keys
cd /etc
cp -r /etc/sshd_config /etc/ssh
echo "Done: cp -r /etc/sshd_config /etc/ssh"
mkdir /etc/openvpn/logs
echo "Done: mkdir /etc/openvpn/logs"
cp -r /etc/logs/server*.log /etc/openvpn/logs
echo "Done: cp -r /etc/logs/server*.log /etc/openvpn/logs"
echo "Done: /etc/setkeys.sh"
/etc/init.d/sshd stop
echo "Done: /etc/init.d/sshd stop"
/etc/init.d/sshd start
echo "Done: /etc/init.d/sshd start"
/etc/init.d/sshd restart
echo "Done: /etc/init.d/sshd restart"
echo "Done.........."
echo "Done.........."
echo "Done.........."
echo "Done.........."

3rd file
Sync.sh (Will be run on Main server)

#!/bin/bash
echo "Main Server xx.xxx.xxx.xx....Source of updated files....."
echo "..................................................................."
echo "..................................................................."
echo "Backup server..........................................................."
echo "VPS xx.xx.xxx.xx Starting to sync accounts......................"
rsync -a -e ssh /etc/passwd root@xx.xx.xxx.xx:/etc/passwd
rsync -a -e ssh /etc/passwd- root@xx.xx.xxx.xx:/etc/passwd-
rsync -a -e ssh /etc/group root@xx.xx.xxx.xx:/etc/group
rsync -a -e ssh /etc/group- root@xx.xx.xxx.xx:/etc/group-
rsync -a -e ssh /etc/shadow root@xx.xx.xxx.xx:/etc/shadow
rsync -a -e ssh /etc/shadow- root@xx.xx.xxx.xx:/etc/shadow-
echo "VPS xx.xx.xxx.xx Done syncing accounts.........................."

I think I manage to run this but It seems its not updating files on the backup server, maybe it wasn't overwriting existing files?

PLease help.!

 

[Zeokat]

The first line of the first script not seems ok. It should be:

#!/bin/bash
cd /etc

Noy sure if the script return any kind of error... honestly i have no idea why it not works.

 

[Lock Down]

First line is fine as it is a comment..

What do you get as far as output when running each of these scripts.

There are many messages displayed.. Use ssh and run each script one at a time and see what errors come up..

 

[collen]

First line is fine as it is a comment..

What do you get as far as output when running each of these scripts.

There are many messages displayed.. Use ssh and run each script one at a time and see what errors come up..

first script don't have any errors.
first script generated the keys and put it on /etc/destinationetc/.ssh/authorized_keys

so I copy /.ssh/authorized_keys and paste it on etc folder on backup server then run second script.

so backup server has now /etc/.ssh/authorized_keys

so when I run the setkeys.sh script (Second Script)

I get this error no file or directory.

SO back to main server to sync the files using THIRD script.
First off I only type in command line /etc/sync.sh and it return an error
"-bash: /etc/sync.sh: /bin/bash^M: bad interpreter: No such file or directory"

So I use a command "sh etc/sync.sh" and here is the result.

When I check the backup server, files still not updated. :-( or nothing has changed.

When sync.sh is run on my main server it says The authenticity of host xxx.xxx.xxx.xx can't be established as per 2nd screenshot.

so I highly think the Second script has the error (my guess would be in the sshd_config thingy?), which is why I think the final script can't go through.

please help sir.

 

[Lock Down]

ok run this for the Sync.sh

#!/bin/bash
echo "Main Server xx.xxx.xxx.xx....Source of updated files....."
echo "..................................................................."
echo "..................................................................."
echo "Backup server..........................................................."
echo "VPS xx.xx.xxx.xx Starting to sync accounts......................"
rsync -[FONT=Verdana]avhe[/FONT][FONT=Verdana] ssh /etc/passwd root@xx.xx.xxx.xx:/etc/passwd[/FONT]
rsync -[FONT=Verdana]avhe[/FONT][FONT=Verdana] ssh /etc/passwd- root@xx.xx.xxx.xx:/etc/passwd-[/FONT]
rsync -[FONT=Verdana]avhe[/FONT][FONT=Verdana] ssh /etc/group root@xx.xx.xxx.xx:/etc/group[/FONT]
rsync -[FONT=Verdana]avhe[/FONT][FONT=Verdana] ssh /etc/group- root@xx.xx.xxx.xx:/etc/group-[/FONT]
rsync -[FONT=Verdana]avhe[/FONT][FONT=Verdana] ssh /etc/shadow root@xx.xx.xxx.xx:/etc/shadow[/FONT]
rsync -[FONT=Verdana]avhe[/FONT][FONT=Verdana] ssh /etc/shadow- root@xx.xx.xxx.xx:/etc/shadow-[/FONT]
echo "VPS xx.xx.xxx.xx Done syncing accounts.........................."

This will tell you what files are being updated if any.. you may have to add the --delete option after -avhe if this does not overwrite them

 

[collen]

ok run this for the Sync.sh

#!/bin/bash
echo "Main Server xx.xxx.xxx.xx....Source of updated files....."
echo "..................................................................."
echo "..................................................................."
echo "Backup server..........................................................."
echo "VPS xx.xx.xxx.xx Starting to sync accounts......................"
rsync -[FONT=Verdana]avhe[/FONT][FONT=Verdana] ssh /etc/passwd root@xx.xx.xxx.xx:/etc/passwd[/FONT]
rsync -[FONT=Verdana]avhe[/FONT][FONT=Verdana] ssh /etc/passwd- root@xx.xx.xxx.xx:/etc/passwd-[/FONT]
rsync -[FONT=Verdana]avhe[/FONT][FONT=Verdana] ssh /etc/group root@xx.xx.xxx.xx:/etc/group[/FONT]
rsync -[FONT=Verdana]avhe[/FONT][FONT=Verdana] ssh /etc/group- root@xx.xx.xxx.xx:/etc/group-[/FONT]
rsync -[FONT=Verdana]avhe[/FONT][FONT=Verdana] ssh /etc/shadow root@xx.xx.xxx.xx:/etc/shadow[/FONT]
rsync -[FONT=Verdana]avhe[/FONT][FONT=Verdana] ssh /etc/shadow- root@xx.xx.xxx.xx:/etc/shadow-[/FONT]
echo "VPS xx.xx.xxx.xx Done syncing accounts.........................."

This will tell you what files are being updated if any.. you may have to add the --delete option after -avhe if this does not overwrite them

not sure if this is the result,
but when checking the files its now doubled.

 

[Lock Down]

So it looks like it is working now. Just add the --delete after rsync. Delete the ones there now and see if it creates new ones. Then run it again in 2 minutes and see if the time is updated.

sample with --delete

rsync --delete -[FONT=Verdana]avhe[/FONT][FONT=Verdana] ssh /etc/passwd [EMAIL="root@xx.xx.xxx.xx"]root@xx.xx.xxx.xx[/EMAIL]:/etc/passwd[/FONT]

 

[collen]

So it looks like it is working now. Just add the --delete after rsync. Delete the ones there now and see if it creates new ones. Then run it again in 2 minutes and see if the time is updated.

sample with --delete

rsync --delete -[FONT=Verdana]avhe[/FONT][FONT=Verdana] ssh /etc/passwd [EMAIL="root@xx.xx.xxx.xx"]root@xx.xx.xxx.xx[/EMAIL]:/etc/passwd[/FONT]

still doubles the files :-( even when --delete is already added.

__________________
Added after 16 Hours 54 minutes:

So it looks like it is working now. Just add the --delete after rsync. Delete the ones there now and see if it creates new ones. Then run it again in 2 minutes and see if the time is updated.

sample with --delete

rsync --delete -[FONT=Verdana]avhe[/FONT][FONT=Verdana] ssh /etc/passwd [EMAIL="root@xx.xx.xxx.xx"]root@xx.xx.xxx.xx[/EMAIL]:/etc/passwd[/FONT]

sir do you think the -a parameter causes this? as the files isn't overwritten as its supposed to be.

 

[Lock Down]

"archive" mode, which ensures that symbolic links, devices, attributes, permissions, ownerships etc are preserved in the transfer.

You can try it without the a but it should not have any effect.

Did you delete all the files on the receiving server before running the rsync? Then try it..

Make sure you do that!

 

[collen]

"archive" mode, which ensures that symbolic links, devices, attributes, permissions, ownerships etc are preserved in the transfer.

You can try it without the a but it should not have any effect.

Did you delete all the files on the receiving server before running the rsync? Then try it..

Make sure you do that!

actually sir I'll be using the script for automation, so I can't log in to other servers (at least 5) delete first the files then run the script on the main server. it would be pointless. won't rsync auto overwrite sir? or what do I need to make it force overwrite please?

 

[Lock Down]

The point of my saying delete those on that one server was to do a test. To see if the already existing files are somehow corrupted and not deleting properly.

Yes they are supposed to be deleting the files on the server that is to get the new files.

So try doing what I asked and see if it corrects the delete problem.. Only delete the files ONCE and try the rsync a couple times to see if you end up with duplicates or or old files.

 

[collen]

The point of my saying delete those on that one server was to do a test. To see if the already existing files are somehow corrupted and not deleting properly.

Yes they are supposed to be deleting the files on the server that is to get the new files.

So try doing what I asked and see if it corrects the delete problem.. Only delete the files ONCE and try the rsync a couple times to see if you end up with duplicates or or old files.

Hi sir, I tried using the rsync on a new fresh installed VPS but still doubles the files. But I think I have near solving the problem found a way but I think I still need a little help sir.

So what I found out is I can remote delete the files using ssh on main server.
command: ssh xxx.xxx.xxx.xx rm /etc/group
command: ssh xxx.xxx.xxx.xx rm /etc/group-

It succesfully deletes the files. BUT the problem is now here sir. When I put the command on the SH file and not run it directly. Example I put the two commands on sync.sh

and when I run sync.sh it shows error (But it is successful when run directly not via SH file)
the error is
rm: cannot remove `/etc/group\r': No such file or directory
rm: cannot remove `/etc/group-\r': No such file or directory

It has \r' or something, not sure why. Maybe that's the reason it shows no such file. But the command is the same on SH file. But don't know why it failed if via sh file and if directly entered on command line it is successful.

__________________
Added after 2 Hours 29 minutes:

I'm getting tired of this.Anyone can fix this for me please. Willing to pay $10 via Paypal.

 

[Lock Down]

Finished. Was asking for a password so used a public key.