Move to Cloud: Difference between revisions

From Adam Colton
Jump to navigation Jump to search
No edit summary
No edit summary
 
(75 intermediate revisions by the same user not shown)
Line 1: Line 1:
I'm giving serious consideration to moving my sites out to AWS. To have a tiny instance running 24/7 will cost $105.
Also see [[Development]]


=Setup=
== Random Notes ==
One tiny instance will run all the live sites, another tiny instance will be used for development. This will also host a Git server.
For wordpress make sure to chown to www-data.


=Comparison=
==Resources==
==Advantages==
* http://calculator.s3.amazonaws.com/calc5.html
* Faster (this is a big problem with D5)
* http://lifehacker.com/5943452/five-best-domain-name-registrars
* Scalable (but do I really need this)
* Isolated Dev environment (and stage if I ever need)
* I can run whatever I want (like Git)


==Disadvantages==
== Migration Steps ==
* Cost (though in reality, I'm already paying this much because of the stupid 'privacy' thing that got turned on'
; AdamColton.net
* Steep learning curve
: WS2
* I will have to transfer all my sites
: up and running, need to fill in
* There is a lot more room for me to screw up
: need to get pics
: transfer to NameCheap (last)


=Plan=
; Wiki
# Get a tiny instance up and running with a lamp stack
: WS2
# Buy a test domain from Amazon and get that hooked up
: need to fix index.php in url
# Terminate it and make sure the EBS volume sticks around
 
# Reattach the volume to a new instance
; Projects
# Try attaching the volume to larger instance.
: WS2
# Transfer the sites over (data)
: not all databases are up and running
# Confirm everything is up, running and works
: move the videos to Vimeo/Youtube
# Transfer domains
 
; Moon-Jumper
: WS1
: always a pain to move
: maybe just start building V3 here, I don't think it will take that long this time around
: transfer to namecheap
 
; AlexColton.net AlexColton.com
: D5
: transfer to namecheap
 
; LaurenColton.com
: Transfer to Lauren
 
==To Do==
*https? - Can do a self-signed cert
*http://aws.amazon.com/articles/1663
* FTP
* Git
* Get MySql Workbench running
* WP with RDS
* sshfs
* Try Dev Environments
** Eclipse
** Vim
** Cloud 9
* SQS
* How to create strict relationships / cascading deletes
* Try AWS load balancer (and look at rates)
* Security for webmin (apparently it's very exposed)
 
Look into component systems
* Testing
* Build/publish
* Bug tracking
* Task tracking
* Source control
 
==Notes==
startup script (for ubuntu) in /etc/init.d/ec2 and to make sure it's run at startup, "update-rc.d ec2 defaults"
 
==Tools and Setup==
=== Server Type ===
I've been using Ubuntu server, and that seems to be work very well. The only quirk (and perhaps not a bad one) is that the system does not allow you to log in as root, you must instead issue sudo.
 
=== Update ===
Do this first. Do it.
 
sudo apt-get update
 
===Server Setup===
Tasksel seems to be a good tool for system setup and is already installed in Ubuntu, just run "sudo tasksel".
 
More here: http://www.unixmen.com/install-lamp-with-1-command-in-ubuntu-1010-maverick-meerkat/
 
=== Webmin ===
Note, the initial attempt to install webmin will fail, but it will tell apt what it needs to finish. When the apt-get -f install command is run, it will get everything and install webmin.
 
http://www.webmin.com/deb.html
 
# wget http://prdownloads.sourceforge.net/webadmin/webmin_1.610_all.deb
# sudo dpkg --install webmin_1.610_all.deb
# sudo apt-get -f install
# sudo ufw allow 10000
# sudo sed -i 's/ssl=1/ssl=0/g' /etc/webmin/miniserv.conf
# sudo /usr/share/webmin/changepass.pl /etc/webmin root PASSWORD
# sudo /etc/webmin/restart
 
=== PhpMyAdmin ===
sudo apt-get install phpmyadmin
 
=== AWS Command Line Tools ===
http://alestic.com/2012/05/aws-command-line-packages
 
=== Attaching Database Volume ===
 
====Through UI====
# Attach a volume through AWS interface
# sudo parted /dev/DEVICE
## mklabel msdo
## mkpart primary 1 -1
## quit
# sudo mkfs.ext4 /dev/DEVICE1
# sudo mkdir -m 000 /vol
# sudo mount /dev/DEVICE1 /vol
# sudo /etc/init.d/mysql stop
# sudo mkdir /vol/etc /vol/lib /vol/log
# sudo mv /etc/mysql    /vol/etc/
# sudo mv /var/lib/mysql /vol/lib/
# sudo mv /var/log/mysql /vol/log/
# sudo mkdir /etc/mysql /var/lib/mysql /var/log/mysql
#sudo /etc/init.d/mysql start
 
====Through CLI====
http://aws.amazon.com/articles/1663
* availabilityZone=`wget -q -O - http://169.254.169.254/latest/meta-data/placement/availability-zone`
* volumeId=`ec2-create-volume -z $availabilityZone -s 8 | grep -Eo 'vol-\w{8}'`
* instanceId=`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`
* ec2-attach-volume $volumeId -d /dev/xvdf -i $instanceId
* parted /dev/xvdf --script -- mklabel msdos
* parted /dev/xvdf --script -- mkpart primary 1 -1
* mkfs.ext4 /dev/xvdf
 
==VNC Setup==
# Update the system
# Install a desktop either through tasksel or sudo apt-get install ubuntu-desktop
# sudo apt-get install vnc4server
# vncserver :1
# vncserver -kill :1 (starting and stopping it generates some necessary files
# sudo pico ~/.vnc/xstartup
#* Erase the last two lines
#* add "gnome-session &" at the end
# sudo passwd ubuntu ALTERNATIVLY try just doing vncpasswd
# vncserver :1
# Open ports 5900-5910
# Connect! I recommend Remmina (or RealVNC for windows)
 
As of 13.04, this doesn't work and you need Gnome-Fallback. As of 13.10, it doesn't work at all.
 
To connect, I needed
  123.123.45.67:1
 
===Blender===
Still don't have this working, but http://www.noobslab.com/2011/09/nvidia-drivers-for-ubuntu-1110-oneiric.html got me past the first set of errors.
 
==Server Setup with Script==
*Upload aws.tar.gz
**pscp -i minecraft.ppk aws.tar.gz ubuntu@54.235.214.84:
**scp -i Minecraft.pem aws.tar.gz ubuntu@54.235.214.84:
*tar -zxvf aws.tar.gz
*wget projects.adamcolton.net/startup/webServer.sh
*sudo chmod u+x webServer.sh
*sudo bash webServer.sh
*source $HOME/.bashrc
 
== PuTTY Notes ==
To make the display cleaner, in Window > Appearance change the font to Courier New and in Window > Translation change the remote character set to UTF-8.
 
== What Needs to Move ==
Before
* projects.adamcolton.net
* wiki.adamcolton.net - DONE
* alexcolton.net and alexcolton.com
* laurencolton.com
* moon-jumper.com - DONE
* Adamcolton.net
 
After
* christmas.adamcolton.net
* miearthship.com
* Move minecraft controller on this server
* Move minecraft wiki on this server
 
=== Steps ===
# Export files - DONE
# Export Databases
# Copy up files and setup up sub-domains
# Copy up Database files
#* adamcolton.net wordpress
#* wiki
#* alexcolton.net wordpress
#* miearthship wordpress
#* moon-jumper databases
# Configure connection strings
# Setup Hosts file and do testing
# Domain transfers
#* Laurens domain gets transferred to her
# Eat a cookie!


[[Category:Projects]]
[[Category:Code]]
[[Category:Code]]

Latest revision as of 15:14, 23 August 2018

Also see Development

Random Notes

For wordpress make sure to chown to www-data.

Resources

Migration Steps

AdamColton.net
WS2
up and running, need to fill in
need to get pics
transfer to NameCheap (last)
Wiki
WS2
need to fix index.php in url
Projects
WS2
not all databases are up and running
move the videos to Vimeo/Youtube
Moon-Jumper
WS1
always a pain to move
maybe just start building V3 here, I don't think it will take that long this time around
transfer to namecheap
AlexColton.net AlexColton.com
D5
transfer to namecheap
LaurenColton.com
Transfer to Lauren

To Do

  • https? - Can do a self-signed cert
  • http://aws.amazon.com/articles/1663
  • FTP
  • Git
  • Get MySql Workbench running
  • WP with RDS
  • sshfs
  • Try Dev Environments
    • Eclipse
    • Vim
    • Cloud 9
  • SQS
  • How to create strict relationships / cascading deletes
  • Try AWS load balancer (and look at rates)
  • Security for webmin (apparently it's very exposed)

Look into component systems

  • Testing
  • Build/publish
  • Bug tracking
  • Task tracking
  • Source control

Notes

startup script (for ubuntu) in /etc/init.d/ec2 and to make sure it's run at startup, "update-rc.d ec2 defaults"

Tools and Setup

Server Type

I've been using Ubuntu server, and that seems to be work very well. The only quirk (and perhaps not a bad one) is that the system does not allow you to log in as root, you must instead issue sudo.

Update

Do this first. Do it.

sudo apt-get update

Server Setup

Tasksel seems to be a good tool for system setup and is already installed in Ubuntu, just run "sudo tasksel".

More here: http://www.unixmen.com/install-lamp-with-1-command-in-ubuntu-1010-maverick-meerkat/

Webmin

Note, the initial attempt to install webmin will fail, but it will tell apt what it needs to finish. When the apt-get -f install command is run, it will get everything and install webmin.

http://www.webmin.com/deb.html

  1. wget http://prdownloads.sourceforge.net/webadmin/webmin_1.610_all.deb
  2. sudo dpkg --install webmin_1.610_all.deb
  3. sudo apt-get -f install
  4. sudo ufw allow 10000
  5. sudo sed -i 's/ssl=1/ssl=0/g' /etc/webmin/miniserv.conf
  6. sudo /usr/share/webmin/changepass.pl /etc/webmin root PASSWORD
  7. sudo /etc/webmin/restart

PhpMyAdmin

sudo apt-get install phpmyadmin

AWS Command Line Tools

http://alestic.com/2012/05/aws-command-line-packages

Attaching Database Volume

Through UI

  1. Attach a volume through AWS interface
  2. sudo parted /dev/DEVICE
    1. mklabel msdo
    2. mkpart primary 1 -1
    3. quit
  3. sudo mkfs.ext4 /dev/DEVICE1
  4. sudo mkdir -m 000 /vol
  5. sudo mount /dev/DEVICE1 /vol
  6. sudo /etc/init.d/mysql stop
  7. sudo mkdir /vol/etc /vol/lib /vol/log
  8. sudo mv /etc/mysql /vol/etc/
  9. sudo mv /var/lib/mysql /vol/lib/
  10. sudo mv /var/log/mysql /vol/log/
  11. sudo mkdir /etc/mysql /var/lib/mysql /var/log/mysql
  12. sudo /etc/init.d/mysql start

Through CLI

http://aws.amazon.com/articles/1663

VNC Setup

  1. Update the system
  2. Install a desktop either through tasksel or sudo apt-get install ubuntu-desktop
  3. sudo apt-get install vnc4server
  4. vncserver :1
  5. vncserver -kill :1 (starting and stopping it generates some necessary files
  6. sudo pico ~/.vnc/xstartup
    • Erase the last two lines
    • add "gnome-session &" at the end
  7. sudo passwd ubuntu ALTERNATIVLY try just doing vncpasswd
  8. vncserver :1
  9. Open ports 5900-5910
  10. Connect! I recommend Remmina (or RealVNC for windows)

As of 13.04, this doesn't work and you need Gnome-Fallback. As of 13.10, it doesn't work at all.

To connect, I needed

 123.123.45.67:1

Blender

Still don't have this working, but http://www.noobslab.com/2011/09/nvidia-drivers-for-ubuntu-1110-oneiric.html got me past the first set of errors.

Server Setup with Script

  • Upload aws.tar.gz
    • pscp -i minecraft.ppk aws.tar.gz ubuntu@54.235.214.84:
    • scp -i Minecraft.pem aws.tar.gz ubuntu@54.235.214.84:
  • tar -zxvf aws.tar.gz
  • wget projects.adamcolton.net/startup/webServer.sh
  • sudo chmod u+x webServer.sh
  • sudo bash webServer.sh
  • source $HOME/.bashrc

PuTTY Notes

To make the display cleaner, in Window > Appearance change the font to Courier New and in Window > Translation change the remote character set to UTF-8.

What Needs to Move

Before

  • projects.adamcolton.net
  • wiki.adamcolton.net - DONE
  • alexcolton.net and alexcolton.com
  • laurencolton.com
  • moon-jumper.com - DONE
  • Adamcolton.net

After

  • christmas.adamcolton.net
  • miearthship.com
  • Move minecraft controller on this server
  • Move minecraft wiki on this server

Steps

  1. Export files - DONE
  2. Export Databases
  3. Copy up files and setup up sub-domains
  4. Copy up Database files
    • adamcolton.net wordpress
    • wiki
    • alexcolton.net wordpress
    • miearthship wordpress
    • moon-jumper databases
  5. Configure connection strings
  6. Setup Hosts file and do testing
  7. Domain transfers
    • Laurens domain gets transferred to her
  8. Eat a cookie!