Move to Cloud: Difference between revisions

From Adam Colton
Jump to navigation Jump to search
No edit summary
 
(55 intermediate revisions by the same user not shown)
Line 1: Line 1:
Currently reading: Programming Amazon EC2.
Also see [[Development]]


==Setup==
== Random Notes ==
One tiny instance will run all the live sites, another tiny instance will be used for development. This will also host a Git server.
For wordpress make sure to chown to www-data.


==Cost==
==Resources==
http://calculator.s3.amazonaws.com/calc5.html
* http://calculator.s3.amazonaws.com/calc5.html
* http://lifehacker.com/5943452/five-best-domain-name-registrars


Just messing around with the calculator, I can get a pretty sweet setup for less than $25 a month. If I run everything on a single micro instance, it's free.
== Migration Steps ==
; AdamColton.net
: WS2
: up and running, need to fill in
: need to get pics
: transfer to NameCheap (last)


==Plan==
; Wiki
# Get a tiny instance up and running with a lamp stack
: WS2
# Buy a test domain from Amazon and get that hooked up
: need to fix index.php in url
# Terminate it and make sure the EBS volume sticks around
# Reattach the volume to a new instance
# Try attaching the volume to larger instance.
# Transfer the sites over (data)
# Confirm everything is up, running and works
# Transfer domains
# Try to figure out SSL


==Also try==
; Projects
*Put the database (data) on a separate EBS volume
: WS2
*Try running that volume on a separate instance (one for apache, one for MySql) - this would allow a lot of scaleability
: not all databases are up and running
*Running FTP server
: move the videos to Vimeo/Youtube
*Running GIT
*AWS command line tools


==Registrars==
; Moon-Jumper
http://lifehacker.com/5943452/five-best-domain-name-registrars
: WS1
: always a pain to move
: maybe just start building V3 here, I don't think it will take that long this time around
: transfer to namecheap


Should move away from Dot 5, just not what I need.
; AlexColton.net AlexColton.com
: D5
: transfer to namecheap


==Things to Learn==
; LaurenColton.com
*How to backup/restore an image
: Transfer to Lauren
*Databases - how do they work? (can I get WP setup with the database on another server, can I move the DB around?)
 
*cpanel, webmin? Needs some research.
==To Do==
*I should really learn vim
*https? - Can do a self-signed cert
*https?
*http://aws.amazon.com/articles/1663
*linux sys admin
* FTP
*if I run my own DB, how does load balancing work (I can always just use the Amazon services)
* Git
*Can I install command line tools on an instance - then I can have one instance controlling others
* Get MySql Workbench running
*How to attach multiple volumes to one machine and where in the filesystem they end up
* WP with RDS
* sshfs
* Try Dev Environments
** Eclipse
** Vim
** Cloud 9
* SQS
* How to create strict relationships / cascading deletes
* Try AWS load balancer (and look at rates)
* Security for webmin (apparently it's very exposed)
 
Look into component systems
* Testing
* Build/publish
* Bug tracking
* Task tracking
* Source control


==Notes==
==Notes==
Line 47: Line 65:


==Tools and Setup==
==Tools and Setup==
=== Server Type ===
I've been using Ubuntu server, and that seems to be work very well. The only quirk (and perhaps not a bad one) is that the system does not allow you to log in as root, you must instead issue sudo.
=== Update ===
=== Update ===
Do this first. Do it.
Do this first. Do it.
sudo apt-get update


===Server Setup===
===Server Setup===
Line 56: Line 79:


=== Webmin ===
=== Webmin ===
# http://www.webmin.com/rpm.html
Note, the initial attempt to install webmin will fail, but it will tell apt what it needs to finish. When the apt-get -f install command is run, it will get everything and install webmin.
 
http://www.webmin.com/deb.html
 
# wget http://prdownloads.sourceforge.net/webadmin/webmin_1.610_all.deb
# sudo dpkg --install webmin_1.610_all.deb
# sudo apt-get -f install
# sudo ufw allow 10000
# sudo ufw allow 10000
# sudo pico miniserv.conf
# sudo sed -i 's/ssl=1/ssl=0/g' /etc/webmin/miniserv.conf
#* change ssl=1 to ssl=0
# sudo /usr/share/webmin/changepass.pl /etc/webmin root PASSWORD
# sudo /usr/share/webmin/changepass.pl /etc/webmin root PASSWORD
# sudo /etc/webmin/stop
# sudo /etc/webmin/restart
# sudo /etc/webmin/start
 
=== PhpMyAdmin ===
sudo apt-get install phpmyadmin
 
=== AWS Command Line Tools ===
http://alestic.com/2012/05/aws-command-line-packages
 
=== Attaching Database Volume ===
 
====Through UI====
# Attach a volume through AWS interface
# sudo parted /dev/DEVICE
## mklabel msdo
## mkpart primary 1 -1
## quit
# sudo mkfs.ext4 /dev/DEVICE1
# sudo mkdir -m 000 /vol
# sudo mount /dev/DEVICE1 /vol
# sudo /etc/init.d/mysql stop
# sudo mkdir /vol/etc /vol/lib /vol/log
# sudo mv /etc/mysql    /vol/etc/
# sudo mv /var/lib/mysql /vol/lib/
# sudo mv /var/log/mysql /vol/log/
# sudo mkdir /etc/mysql /var/lib/mysql /var/log/mysql
#sudo /etc/init.d/mysql start
 
====Through CLI====
http://aws.amazon.com/articles/1663
* availabilityZone=`wget -q -O - http://169.254.169.254/latest/meta-data/placement/availability-zone`
* volumeId=`ec2-create-volume -z $availabilityZone -s 8 | grep -Eo 'vol-\w{8}'`
* instanceId=`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`
* ec2-attach-volume $volumeId -d /dev/xvdf -i $instanceId
* parted /dev/xvdf --script -- mklabel msdos
* parted /dev/xvdf --script -- mkpart primary 1 -1
* mkfs.ext4 /dev/xvdf
 
==VNC Setup==
# Update the system
# Install a desktop either through tasksel or sudo apt-get install ubuntu-desktop
# sudo apt-get install vnc4server
# vncserver :1
# vncserver -kill :1 (starting and stopping it generates some necessary files
# sudo pico ~/.vnc/xstartup
#* Erase the last two lines
#* add "gnome-session &" at the end
# sudo passwd ubuntu ALTERNATIVLY try just doing vncpasswd
# vncserver :1
# Open ports 5900-5910
# Connect! I recommend Remmina (or RealVNC for windows)
 
As of 13.04, this doesn't work and you need Gnome-Fallback. As of 13.10, it doesn't work at all.
 
To connect, I needed
  123.123.45.67:1
 
===Blender===
Still don't have this working, but http://www.noobslab.com/2011/09/nvidia-drivers-for-ubuntu-1110-oneiric.html got me past the first set of errors.
 
==Server Setup with Script==
*Upload aws.tar.gz
**pscp -i minecraft.ppk aws.tar.gz ubuntu@54.235.214.84:
**scp -i Minecraft.pem aws.tar.gz ubuntu@54.235.214.84:
*tar -zxvf aws.tar.gz
*wget projects.adamcolton.net/startup/webServer.sh
*sudo chmod u+x webServer.sh
*sudo bash webServer.sh
*source $HOME/.bashrc
 
== PuTTY Notes ==
To make the display cleaner, in Window > Appearance change the font to Courier New and in Window > Translation change the remote character set to UTF-8.
 
== What Needs to Move ==
Before
* projects.adamcolton.net
* wiki.adamcolton.net - DONE
* alexcolton.net and alexcolton.com
* laurencolton.com
* moon-jumper.com - DONE
* Adamcolton.net
 
After
* christmas.adamcolton.net
* miearthship.com
* Move minecraft controller on this server
* Move minecraft wiki on this server
 
=== Steps ===
# Export files - DONE
# Export Databases
# Copy up files and setup up sub-domains
# Copy up Database files
#* adamcolton.net wordpress
#* wiki
#* alexcolton.net wordpress
#* miearthship wordpress
#* moon-jumper databases
# Configure connection strings
# Setup Hosts file and do testing
# Domain transfers
#* Laurens domain gets transferred to her
# Eat a cookie!


[[Category:Projects]]
[[Category:Code]]
[[Category:Code]]

Latest revision as of 15:14, 23 August 2018

Also see Development

Random Notes

For wordpress make sure to chown to www-data.

Resources

Migration Steps

AdamColton.net
WS2
up and running, need to fill in
need to get pics
transfer to NameCheap (last)
Wiki
WS2
need to fix index.php in url
Projects
WS2
not all databases are up and running
move the videos to Vimeo/Youtube
Moon-Jumper
WS1
always a pain to move
maybe just start building V3 here, I don't think it will take that long this time around
transfer to namecheap
AlexColton.net AlexColton.com
D5
transfer to namecheap
LaurenColton.com
Transfer to Lauren

To Do

  • https? - Can do a self-signed cert
  • http://aws.amazon.com/articles/1663
  • FTP
  • Git
  • Get MySql Workbench running
  • WP with RDS
  • sshfs
  • Try Dev Environments
    • Eclipse
    • Vim
    • Cloud 9
  • SQS
  • How to create strict relationships / cascading deletes
  • Try AWS load balancer (and look at rates)
  • Security for webmin (apparently it's very exposed)

Look into component systems

  • Testing
  • Build/publish
  • Bug tracking
  • Task tracking
  • Source control

Notes

startup script (for ubuntu) in /etc/init.d/ec2 and to make sure it's run at startup, "update-rc.d ec2 defaults"

Tools and Setup

Server Type

I've been using Ubuntu server, and that seems to be work very well. The only quirk (and perhaps not a bad one) is that the system does not allow you to log in as root, you must instead issue sudo.

Update

Do this first. Do it.

sudo apt-get update

Server Setup

Tasksel seems to be a good tool for system setup and is already installed in Ubuntu, just run "sudo tasksel".

More here: http://www.unixmen.com/install-lamp-with-1-command-in-ubuntu-1010-maverick-meerkat/

Webmin

Note, the initial attempt to install webmin will fail, but it will tell apt what it needs to finish. When the apt-get -f install command is run, it will get everything and install webmin.

http://www.webmin.com/deb.html

  1. wget http://prdownloads.sourceforge.net/webadmin/webmin_1.610_all.deb
  2. sudo dpkg --install webmin_1.610_all.deb
  3. sudo apt-get -f install
  4. sudo ufw allow 10000
  5. sudo sed -i 's/ssl=1/ssl=0/g' /etc/webmin/miniserv.conf
  6. sudo /usr/share/webmin/changepass.pl /etc/webmin root PASSWORD
  7. sudo /etc/webmin/restart

PhpMyAdmin

sudo apt-get install phpmyadmin

AWS Command Line Tools

http://alestic.com/2012/05/aws-command-line-packages

Attaching Database Volume

Through UI

  1. Attach a volume through AWS interface
  2. sudo parted /dev/DEVICE
    1. mklabel msdo
    2. mkpart primary 1 -1
    3. quit
  3. sudo mkfs.ext4 /dev/DEVICE1
  4. sudo mkdir -m 000 /vol
  5. sudo mount /dev/DEVICE1 /vol
  6. sudo /etc/init.d/mysql stop
  7. sudo mkdir /vol/etc /vol/lib /vol/log
  8. sudo mv /etc/mysql /vol/etc/
  9. sudo mv /var/lib/mysql /vol/lib/
  10. sudo mv /var/log/mysql /vol/log/
  11. sudo mkdir /etc/mysql /var/lib/mysql /var/log/mysql
  12. sudo /etc/init.d/mysql start

Through CLI

http://aws.amazon.com/articles/1663

VNC Setup

  1. Update the system
  2. Install a desktop either through tasksel or sudo apt-get install ubuntu-desktop
  3. sudo apt-get install vnc4server
  4. vncserver :1
  5. vncserver -kill :1 (starting and stopping it generates some necessary files
  6. sudo pico ~/.vnc/xstartup
    • Erase the last two lines
    • add "gnome-session &" at the end
  7. sudo passwd ubuntu ALTERNATIVLY try just doing vncpasswd
  8. vncserver :1
  9. Open ports 5900-5910
  10. Connect! I recommend Remmina (or RealVNC for windows)

As of 13.04, this doesn't work and you need Gnome-Fallback. As of 13.10, it doesn't work at all.

To connect, I needed

 123.123.45.67:1

Blender

Still don't have this working, but http://www.noobslab.com/2011/09/nvidia-drivers-for-ubuntu-1110-oneiric.html got me past the first set of errors.

Server Setup with Script

  • Upload aws.tar.gz
    • pscp -i minecraft.ppk aws.tar.gz ubuntu@54.235.214.84:
    • scp -i Minecraft.pem aws.tar.gz ubuntu@54.235.214.84:
  • tar -zxvf aws.tar.gz
  • wget projects.adamcolton.net/startup/webServer.sh
  • sudo chmod u+x webServer.sh
  • sudo bash webServer.sh
  • source $HOME/.bashrc

PuTTY Notes

To make the display cleaner, in Window > Appearance change the font to Courier New and in Window > Translation change the remote character set to UTF-8.

What Needs to Move

Before

  • projects.adamcolton.net
  • wiki.adamcolton.net - DONE
  • alexcolton.net and alexcolton.com
  • laurencolton.com
  • moon-jumper.com - DONE
  • Adamcolton.net

After

  • christmas.adamcolton.net
  • miearthship.com
  • Move minecraft controller on this server
  • Move minecraft wiki on this server

Steps

  1. Export files - DONE
  2. Export Databases
  3. Copy up files and setup up sub-domains
  4. Copy up Database files
    • adamcolton.net wordpress
    • wiki
    • alexcolton.net wordpress
    • miearthship wordpress
    • moon-jumper databases
  5. Configure connection strings
  6. Setup Hosts file and do testing
  7. Domain transfers
    • Laurens domain gets transferred to her
  8. Eat a cookie!