Ngo Duy Khanh's Blog | Learn IT – Share IT || Quản trị mạng || Hạ tầng mạng || Bảo mật mạng || Máy chủ || Windows Server || Cluster

Cấu hình Proxmox server sử dụng một Public IP cho tất cả VM

Đăng vào ngày 2 Tháng Sáu 2013 lúc 12:41 bởi Duy Khánh Mục: IT, Unix/Linux, Virtualization  ||  Tags: , ,    Bình luận (0)

I - Cài đặt và cấu hình Proxmox:

- Download file ISO tại trang chủ: http://www.proxmox.com

- Sau khi cài đặt thành công proxmox, chúng ta có thể truy cập vào giao diện web tại địa chỉ https://<ip-server>:8006

- Mặc định sau khi cài đặt xong Proxmox địa chỉ IP public sẽ được gán trên card vmbr0, card này được bridge với card eth0 của server. Chúng ta cần thay đổi một số thông tin tại đây.

/etc/network/interfaces

Địa chỉ IP public của server sẽ được gán trên card eth0. Đối với card vmbr0 chúng ta sẽ chuyển bridge_ports sang none (thay vì eth0) trước đó. Cấu hình mẫu như bên dưới

# network interface settings
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address  192.168.100.115
        netmask  255.255.255.0
        gateway  192.168.100.2
        dns-nameservers 8.8.8.8

auto vmbr0
iface vmbr0 inet static
        address  10.10.10.1
        netmask  255.255.255.0
        bridge_ports none
        bridge_stp off
        bridge_fd 0

Giả sử ở cấu hình trên, địa chỉ public của server sẽ là 192.168.100.115 được gán lên card eth0, còn địa chỉ 10.10.10.1 được gán trên card vmbr0 sẽ giao tiếp với các VM bên trong.

Trên giao diện web của Proxmox, thẻ network ta sẽ có thông tin như sau

Chúng ta cũng sẽ setup một VM để test. VM này sẽ có địa chỉ IP là 10.10.10.10.

 

II - Cài đặt và cấu hình shorewall

- Cài đặt shorewall bằng lệnh: apt-get install shorewall

- Kích hoạt Ip forwarding trên shorewall bằng cách mở file /etc/shorewall/shorewall.conf

Đổi

IP_FORWARDING=Off

Thành

IP_FORWARDING=On

- Cấu hình Zone trong file /etc/shorewall/zones. Chúng ta sẽ có 3 Zone fw, net, và dmz. Trong đó net là zone giao tiếp từ môi trường internet, dmz là zone nội bộ bên trong.

#ZONE   TYPE            OPTIONS         IN                      OUT
#                                       OPTIONS                 OPTIONS
fw      firewall
net     ipv4
dmz     ipv4

- Cho biết các zone được kết nối thông qua interface nào. Tạo file /etc/shorewall/interfaces

#ZONE   INTERFACE       BROADCAST       OPTIONS
net     eth0            detect          blacklist,nosmurfs
dmz     venet0          detect          routeback
dmz     vmbr0           detect          routeback,bridge

- Tiếp tục, chúng ta cấu hình các policy mặc định giữa các zone. Tạo file /etc/shorewall/policy

#SOURCE DEST    POLICY          LOG     LIMIT:          CONNLIMIT:
#                               LEVEL   BURST           MASK

# From Firewall Policy
fw      fw      ACCEPT
fw      net     ACCEPT
fw      dmz     ACCEPT

# From DMZ Policy

dmz     dmz     ACCEPT
dmz     net     ACCEPT
dmz     fw      DROP            info

# From Net Policy
net     fw      DROP            info
net     dmz     DROP            info 

# THE FOLLOWING POLICY MUST BE LAST
#
all     all     REJECT          info

- Thiết lập rules cho shorewall tại /etc/shorewall/rules

#ACTION          SOURCE     DEST       PROTO   DEST        SOURCE     ORIGINAL    RATE

# Permit access to SSH
SSH/ACCEPT       net        fw         -       -            -          -          6/min:5

# Permit access to Proxmox Manager and Console
ACCEPT           net        fw                        tcp     8006,5900:5999
# VM access rules DNAT net dmz:10.10.10.10:22 tcp 1022 # PING Rules Ping/ACCEPT all all # LAST LINE -- DO NOT REMOVE

Trong ví dụ về cấu hình rule ở trên, chúng ta cho phép SSH và ping tới Proxmox server. Ngoài ra, chúng ta còn thực hiện việc NAT port 22 trên VM. Với cấu hình này, nếu chúng ta muốn SSH tới VM từ bên ngoài internet, chúng ta có thể SSH thông qua public IP với port là 1022 (Cấu hình cho các dịch vụ khác tương tự).

- Thiết lập cho phép kết nối từ VM ra ngoài internet thông qua tệp tin /etc/shorewall/masq

#INTERFACE      SOURCE          ADDRESS         PROTO   PORT(S) IPSEC   MARK
eth0            10.10.10.0/24

# LAST LINE -- DO NOT REMOVE

- Kiểm tra cấu hình của shorewall đã đúng hay chưa

shorewall check
- Nếu mọi cấu hình đều OK, chúng ta kích hoạt cho phép Shorewall được khởi động
/etc/default/shorewall
- Khởi động Shorewall
service shorewall start
Chú ý: Nếu bạn SSH tới Proxmox server từ xa. Việc cấu hình sai shorewall có thể làm bạn bị mất kết nối khi khởi động dịch vụ shorewall. Để tránh tình trạng này, chúng ta có thể sử dụng lệnh sau để kiểm tra trước. Với lệnh này, shorewall sẽ chỉ apply cấu hình trong vòng 60 giây, sau thời gian đó, nó sẽ tự động khôi phục lại trạng thái trước đó.
shorewall try /etc/shorewall 60
Mọi cấu hình đã xong, bây giờ chúng ta có thể truy xất SSH tới VM bên trong thông qua ip public của Proxmox Server.

Install puppet on ubuntu server

Đăng vào ngày 26 Tháng Năm 2013 lúc 22:24 bởi Duy Khánh Mục: IT, Unix/Linux  ||  Tags: , ,    Bình luận (0)

Install puppet master

master$ sudo apt-get -y install puppetmaster

Regenerate Master Certificate

master$ sudo service puppetmaster stop

master$ sudo rm -r /var/lib/puppet/ssl

Add master’s name to config

master$ sudoedit /etc/puppet/puppet.conf

add these names under [master] heading

dns_alt_names = puppet, master.local, puppet.khanh.com.vn

Certificate is automatically generated when you start PuppetMaster

master$ sudo service puppetmaster start

Connect from Slave

slave$ sudo apt-get -y install puppet

slave$ sudoedit /etc/puppet/puppet.conf

Add master DNS name under [agent] heading. Puppet will connect to server.

[agent]

server = master.local

Allow puppet slave to start

slave$ sudoedit /etc/default/puppet

Change to yes:

START=yes

Sign Slave Certificate on Master

master$ sudo puppet cert --list

master$ sudo puppet cert --sign slave.example.com

Create Site Manifest and a Module

....

Test Slave get configuration from Master

slave$ sudo puppet agent --test

------------------------

Force agent to regenerate certificate request by generate command

 

[puppet agent]$ puppet  certificate   generate    web1  --ca-location  remote
warning: peer certificate won't be verified in this SSL session
err: Error 400 on SERVER: web1 already has a revoked certificate; ignoring certificate request
err: Try 'puppet help certificate generate' for usage
#It because the revoked certificate still exist in the server, it need to be deleted
[puppet master]$ puppet cert list –all
- web1                                     (BA:18:D1:86:D6:5E:9E:99:55:39:3D:67:79:BF:BD:D0) (certificate revoked)
[puppet master]$ puppet cert clean web1
#re-run the command, the warning is expected because the request hasn’t been signed by master yet
[puppet agent]$puppet certificate generate web1 --ca-location remote
warning: peer certificate won't be verified in this SSL session
true
#The pending request appears in master 
[puppet master]$ puppet cert list
web1 (3B:ED:D9:8D:2F:C2:A1:D3:89:B4:D0:FD:41:7E:5E:0C)
#Sign the certificate
[puppet master]# puppet cert sign web1

Error "Unable to read data from the Metabase. Ensure that Microsoft Internet Information Services is installed" while installing Exchange 2010 on W2K8

Đăng vào ngày 19 Tháng Năm 2013 lúc 11:17 bởi Duy Khánh Mục: IT, Mail Server, Windows Server, Troubleshooting  ||  Tags: ,    Bình luận (0)

 

PROBLEM DESCRIPTION:

While installing Exchange 2010 on Windows 2008 R2 some times admins can see two error messages

Errors:

- Unable to read data from the Metabase. Ensure that Microsoft Internet Information Services is installed.

- Unable to access the ‘Default Web Site’ on this computer.

CAUSE: Exchange 2010 Prerequisites are not met successfully

SOLUTION: One can follow Microsoft Article to make sure all prerequisites are met:

http://technet.microsoft.com/en-us/library/bb691354(EXCHG.140).aspx

If all common roles are being installed on one server as I had to do some time back for test environment following is what is needed to be executed from Windows Powershell (should be started with "Run As Administrator")

- Import-Module ServerManager

- Add-WindowsFeature NET-Framework,RSAT-ADDS,Web-Server,Web-Basic-Auth,Web-Windows-Auth,Web-Metabase,Web-Net-Ext,Web-Lgcy-Mgmt-Console,WAS-Process-Model,RSAT-Web-Server,Web-ISAPI-Ext,Web-Digest-Auth,Web-Dyn-Compression,NET-HTTP-Activation,RPC-Over-HTTP-Proxy -Restart

- Set-Service NetTcpPortSharing -StartupType Automatic

Install sysbench on CentOS

Đăng vào ngày 24 Tháng Tư 2013 lúc 22:45 bởi Duy Khánh Mục: IT, SQL, Unix/Linux  ||  Tags: , ,    Bình luận (0)

Download sysbench from: http://sourceforge.net/projects/sysbench/

Sysbench need mysql-devel installed

# yum install mysql-devel

Following commands to install sysbench

# aclocal

# libtoolize --force --copy

sysbench_installation_root_folder >  ./autogen.sh

sysbench_installation_root_folder > ./configure

sysbench_installation_root_folder > make

sysbench_installation_root_folder > make install 

 

Example: Benchmark MySQL

1. Create test table in database test with 1.000.000 records

# sysbench --test=oltp --oltp-table-size=1000000 --mysql-db=test --mysql-user=root --mysql-password=mypassword prepare

2. Benchmark MySQL follow command

# sysbench --test=oltp --oltp-table-size=1000000 --mysql-db=test --mysql-user=root --mysql-password=mypassword --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run

3. Clean up DB afterwards

# sysbench --test=oltp --mysql-db=test --mysql-user=root --mysql-password=mypassword cleanup

Chạy nhiều Tomcat instance trên cùng một server

Đăng vào ngày 28 Tháng Ba 2013 lúc 07:42 bởi Duy Khánh Mục: IT, Unix/Linux, Web Design  ||  Tags: , , ,    Bình luận (1)

B1: Cài đặt Tomcat

Cài đặt Tomcat phân chia thành các thư mục khác nhau, ví dụ /var/tomcat/serverA /var/tomcat/serverB. Trong đó các thư mục serverA, serverB cần có những thư mục nội dung và cấu hình sau: bin, lib, conf, logs, temp, webapps, work

Xem thêm: Cách cài đặt Tomcat7

B2: Thay đổi port

Mỗi Instance sẽ lắng nghe trên các port khác nhau, cần phải thay đổi để không bị xung đột. Có 3 port cần phải đổi là

  • shutdown port: Port này được dùng bở tiến trình java khi tomcat chạy. Khi script shutdown.sh được thực hiện, tomcat sẽ gửi tín hiện kết thúc thông qua port này. Ví dụ 8005
  • connector port: Port dùng để kết nối tới tomcat từ xa (trình duyệt). Ví dụ 8080
  • ajp port: Port dùng khi apache httpd server liên lạc với tomcat, sử dụng để tạo loadbalancer. Ví dụ: 8009

Bạn cần sử file conf/server.xml thay đổi port

B3: Tạo init script cho mỗi Instance

 

#!/bin/bash

    # description: Tomcat Start Stop Restart

    # processname: tomcat

    # chkconfig: 234 20 80

    JAVA_HOME=/usr/java/jdk1.7.0_17/

    export JAVA_HOME

    PATH=$JAVA_HOME/bin:$PATH

    export PATH

    CATALINA_HOME=/usr/share/tomcat7

 

    case $1 in

    start)

    sh $CATALINA_HOME/bin/startup.sh

    ;;

    stop)

    sh $CATALINA_HOME/bin/shutdown.sh

    ;;

    restart)

    sh $CATALINA_HOME/bin/shutdown.sh

    sh $CATALINA_HOME/bin/startup.sh

    ;;

    esac

    exit 0

 

------------------------

 

 

#!/bin/bash

    # description: Tomcat Start Stop Restart

    # processname: tomcat

    # chkconfig: 234 20 80

    JAVA_HOME=/usr/java/jdk1.7.0_17/

    export JAVA_HOME

    PATH=$JAVA_HOME/bin:$PATH

    export PATH

    CATALINA_BASE=/usr/share/tomcat7-new

 

    case $1 in

    start)

    sh $CATALINA_BASE/bin/startup.sh

    ;;

    stop)

    sh $CATALINA_BASE/bin/shutdown.sh

    ;;

    restart)

    sh $CATALINA_BASE/bin/shutdown.sh

    sh $CATALINA_BASE/bin/startup.sh

    ;;

    esac

    exit 0

Install Tomcat 7 on Centos 6.3

Đăng vào ngày 16 Tháng Ba 2013 lúc 07:46 bởi Duy Khánh Mục: IT, Unix/Linux, Web Design  ||  Tags: , , , ,    Bình luận (1)

This post will cover installing and basic configuration of apache-tomcat-7.0.30

If you do not already have the Java Development Kit (JDK) installed on your machine, you will need to download and install the required JDK for your platform.

If you do have the JDK installed, you can skip to: Step 2: Download and Install the  apache-tomcat-7.0.30

Step 1: Install the JDK
[root@station1 Downloads]# rpm -ivh jdk-7u7-linux-x64.rpm

--> 

[root@station1 Downloads]# JAVA_HOME=/usr/java/jdk1.7.0_17
[root@station1 Downloads]# export JAVA_HOME
[root@station1 Downloads]# PATH=$JAVA_HOME/bin:$PATH
[root@station1 Downloads]# export PATH 
[root@station1 Downloads]# vim /root/.bash_profile
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

# User specific environment and startup programs
JAVA_HOME=/usr/java/jdk1.7.0_17/
export JAVA_HOME
PATH=$JAVA_HOME/bin:$PATH
export PATH
#PATH=$PATH:$HOME/bin
#export PATH
Restart or logout and login
[root@station1 Downloads]# echo $JAVA_HOME
/usr/java/jdk1.7.0_17/
       Step 2: Download and Unpack apache-tomcat-7.0.30 (or latest)
To download apache-tomcat click here
[root@station1 Downloads]# tar -xzvf apache-tomcat-7.0.30.tar.gz 
[root@station1 Downloads]# mv apache-tomcat-7.0.30 /usr/share/ 
[root@station1 Downloads]# vim /etc/init.d/tomcat 
    #!/bin/bash
    # description: Tomcat Start Stop Restart
    # processname: tomcat
    # chkconfig: 234 20 80
    JAVA_HOME=/usr/java/jdk1.7.0_17
    export JAVA_HOME
    PATH=$JAVA_HOME/bin:$PATH
    export PATH
    CATALINA_HOME=/usr/share/apache-tomcat-7.0.30

    case $1 in
    start)
    sh $CATALINA_HOME/bin/startup.sh
    ;;
    stop)
    sh $CATALINA_HOME/bin/shutdown.sh
    ;;
    restart)
    sh $CATALINA_HOME/bin/shutdown.sh
    sh $CATALINA_HOME/bin/startup.sh
    ;;
    esac
    exit 0
[root@station1 ~]# chmod 755 /etc/init.d/tomcat
[root@station1 ~]# chkconfig --add tomcat
[root@station1 ~]# chkconfig --level 234 tomcat on
[root@station1 ~]# chkconfig --list tomcat
tomcat             0:off    1:off    2:on    3:on    4:on    5:off    6:off
[root@station1 ~]# service tomcat stop
Using CATALINA_BASE:   /usr/share/apache-tomcat-7.0.30
Using CATALINA_HOME:   /usr/share/apache-tomcat-7.0.30
Using CATALINA_TMPDIR: /usr/share/apache-tomcat-7.0.30/temp
Using JRE_HOME:        /usr/java/jdk1.7.0_17
Using CLASSPATH:       /usr/share/apache-tomcat-7.0.30/bin/bootstrap.jar:/usr/share/apache-tomcat-7.0.30/bin/tomcat-juli.jar
[root@station1 ~]# service tomcat start
Using CATALINA_BASE:   /usr/share/apache-tomcat-7.0.30
Using CATALINA_HOME:   /usr/share/apache-tomcat-7.0.30
Using CATALINA_TMPDIR: /usr/share/apache-tomcat-7.0.30/temp
Using JRE_HOME:        /usr/java/jdk1.7.0_17
Using CLASSPATH:       /usr/share/apache-tomcat-7.0.30/bin/bootstrap.jar:/usr/share/apache-tomcat-7.0.30/bin/tomcat-juli.jar
[root@station1 ~]# service tomcat restart
Using CATALINA_BASE:   /usr/share/apache-tomcat-7.0.30
Using CATALINA_HOME:   /usr/share/apache-tomcat-7.0.30
Using CATALINA_TMPDIR: /usr/share/apache-tomcat-7.0.30/temp
Using JRE_HOME:        /usr/java/jdk1.7.0_17
Using CLASSPATH:       /usr/share/apache-tomcat-7.0.30/bin/bootstrap.jar:/usr/share/apache-tomcat-7.0.30/bin/tomcat-juli.jar
Using CATALINA_BASE:   /usr/share/apache-tomcat-7.0.30
Using CATALINA_HOME:   /usr/share/apache-tomcat-7.0.30
Using CATALINA_TMPDIR: /usr/share/apache-tomcat-7.0.30/temp
Using JRE_HOME:        /usr/java/jdk1.7.0_07
Using CLASSPATH:       /usr/share/apache-tomcat-7.0.30/bin/bootstrap.jar:/usr/share/apache-tomcat-7.0.30/bin/tomcat-juli.jar
For logs

[root@station1 ~]# less /usr/share/apache-tomcat-7.0.30/logs/catalina.out
We can now access the Tomcat Manager page at http://127.0.0.1:8080
------------

Fedora - How to change boot sequence in Grub2

Đăng vào ngày 16 Tháng Ba 2013 lúc 07:40 bởi Duy Khánh Mục: IT, Unix/Linux  ||  Tags: , ,    Bình luận (2)

his is for dual boot os users, there are many who prefers dual boot to keep another OS optional. For any reason if you wish to change boot sequence unlike grub, grub2 requires couple of commands to run.

Here will take example to keep Windows as default boot.
First of all find menuentry for Windows

cat/boot/grub2/grub.cfg |grep Windows

result:-

menuentry "Windows 7 (loader) (on /dev/sda1)"--class windows --class os {

Set Windows menuentry as default (only entry mentioned in either " or 'from above command)

grub2-set-default "Windows 7 (loader) (on /dev/sda1)"

Verify default entry

grub2-editenv list

Generate updated grub.cfg

grub2-mkconfig -o/boot/grub2/grub.cfg

Below is to set Fedora as default boot

cat /boot/grub2/grub.cfg |grep Fedora
grub2-set-default "Fedora Linux, with Linux 3.1.0-5.fc16.i686"
grub2-editenv list
grub2-mkconfig -o /boot/grub2/grub.cfg

 

 

Extending a logical volume in a virtual machine running Red Hat or Cent OS

Đăng vào ngày 4 Tháng Hai 2013 lúc 20:38 bởi Duy Khánh Mục: Unix/Linux, VMware  ||  Tags: , , ,    Bình luận (1)

 

Note: These steps only apply to EXT3 file systems.
 
Caution: VMware recommends having a complete backup of the virtual machine prior making these changes.
  1. Power off the virtual machine.
  2. Edit the virtual machine settings and extend the virtual disk size. For more information, see Increasing the size of a virtual disk (1004047).
  3. Power on the virtual machine.
  4. Identify the device name, which is by default /dev/sda, and confirm the new size by running the command:

    # fdisk -l

  5. Create a new primary partition:
    1. Run the command:

      # fdisk /dev/sda (depending the results of the step 4)
       
    2. Press p to print the partition table to identify the number of partitions. By default there are 2: sda1 and sda2.
    3. Press n to create a new primary partition. 
    4. Press p for primary.
    5. Press 3 for the partition number, depending the output of the partition table print.
    6. Press Enter two times.
    7. Press w to write the changes to the partition table.

  6. Restart the virtual machine.
  7. Run this command to verify that the changes were saved to the partition table and that the new partition has an 83 type:

    # fdisk -l

  8. Run this command to convert the new partition to a physical volume:

    # pvcreate /dev/sda3

  9. Run this command to extend the physical volume:

    # vgextend VolGroup00 /dev/sda3

    Note: To determine which volume group to extend, use the command vgdisplay.

  10. Run this command to verify how many physical extents are available to the Volume Group:

    # vgdisplay VolGroup00 | grep "Free"

  11. Run the following command to extend the Logical Volume:

    # lvextend -L+#G /dev/VolGroup00/LogVol00

    Where # is the number of Free space in GB available as per the previous command.

    Note: to determine which logical volume to extend, use the command lvdisplay.

  12. Run the following command to expand the ext3 filesystem online, inside of the Logical Volume:

    # ext2online /dev/VolGroup00/LogVol00

    Note: Use resize2fs instead of ext2online if it is not a Red Hat virtual machine.

  13. Run the following command to verify that the / filesystem has the new space available:

    # df -h /

 

Live Migration on vSphere 5.1 - Hyper-V 3.0 - XenServer 6.1

Đăng vào ngày 26 Tháng Chín 2012 lúc 15:01 bởi Duy Khánh Mục: IT, Virtualization, VMware  ||  Tags: , ,    Bình luận (1)

vSphere 5.1: vMotion without Shared Storage

 

Windows Server 2012 & Hyper-V 3.0 Live Migration

 

XenServer Storage Motion (block migration) on OpenStack

The Case for Larger Than 2TB Virtual Disks and The Gotcha with VMFS

Đăng vào ngày 21 Tháng Chín 2012 lúc 19:37 bởi Duy Khánh Mục: IT, Virtualization, VMware  ||  Tags: ,    Bình luận (1)

Hypervisor competition is really starting to heat up. VMware just released vSphere 5.1 and Microsoft has recently released Windows Server 2012 and the new version of Hyper-V. A significant  new feature available now in Hyper-V / Windows 2012 is a new disk format VHDX, which has a maximum size of 64TB. With the new filesystem in Windows Server 2012 (ReFS) the maximum volume size increases to 256TB ( NTFS was limited to 16TB @ 4K cluster size). So how does vSphere 5 and 5.1 compare and what are the key considerations and gotchas? What are the implications for business critical applications? Read on to find out.

 

Before we get started I’d like to say this article isn’t going to cover performance of large volumes. But rather the argument for supporting larger than 2TB individual virtual disks and large volumes. There are many considerations around performance, and I will cover some of the implications when you start to scale up volume size, but for particular performance design considerations I’d like to recommend you read my article titled Storage Sizing Considerations when Virtualizing Business Critical Applications.

The Case for Larger than 2TB Virtual Disks

Recently I have been having an interesting debate with some of my VCDX peers on the merits and reasons for having larger than 2TB virtual disk support in vSphere. As of vSphere 5 VMware supports 64TB VMFS5 datastores, and 64TB Physical Mode (Pass-through) Raw Device Maps (RDM’s), but the largest single VMDK file supported on a VMFS5 volume is still 2TB-512b (hereon after referred to as 2TB). The same 2TB limit applies to virtual mode RDMs also. In this debate I’ve been suggesting that for now “most” applications can be supported with the 2TB virtual disk limit. If larger than 2TB volumes are required for a VM that is very easily accommodated with in guest volume managers and device concatenation of multiple 2TB disks, or using an alternative to VMFS. However realistically this can only go so far. I plan to cover both the pros and the cons as I see them.

Pros:

  • Support for an individual VM with larger than 120TB storage requirements, which is the theoretical limit with 4 x vSCSI controllers, each with 15 disks (60 disks total) at the  maximum size of 2TB each. You’ll find out why it’s a theoretical limit later.
  • Easier to manage less devices and less volumes and space can potentially be more efficiently utilised.
  • No need to use in guest volume managers for very large volumes.
  • Easier to support very large individual files >2TB without the use of in guest volume managers.
  • It could be argued that losing one 2TB device from a in guest managed volume has the same risk profile as losing a single large volume of the same size as in both cases the entire volume is potentially lost.

Cons:

  • Larger individual devices and volumes take longer to backup and restore. This may require a major change in data protection architecture.
  • Larger volumes will potentially take longer to replicate and recover in a DR scenario.
  • The risk profile of losing a large volume or device is significantly higher than losing a smaller device or volume. Losing a single smaller device where no volume manager is being used results in only the small device having to be recovered instead of everything.
  • Larger individual devices still have the same number of IO queues to the vSCSI controller which effectively limits their performance. This increases the risk of running out of performance before running out of capacity (until ultra low latency solid state flash storage is of massive capacity and abundantly available anyway).
  • Significantly harder to take snapshots. A snapshot could still grow to be equally as large as the original virtual disk. This is probably one of the more significant reasons that VMware hasn’t yet introduced VMDK’s above 2TB.
  • Significantly longer to check disk for integrity if there is any type of corruption, how will it be recovered if it’s very large?
  • Impact on Storage vMotion times.

In my opinion the arguments are pretty even. But as I always err on the side of performance, and I think having more devices of a smaller size in a lot of cases is a better option as this gives you far more access to more queues and more parallel IO channels. However this is only relevant for some applications, mostly OLTP and messaging type applications. File servers, data warehousing, big data and the like may well benefit greatly from larger volume sizes, and it would make those applications significantly easier to manage. But the requirements will all be driven by the applications and at the moment I only see a very small minority of workloads require storage capacities that would justify very large individual SCSI devices and where the performance tradeoffs from an IO parallelism perspective are acceptable. Most of those corner cases have a suitable alternative for now (discussed below).

Options for Larger than 2TB Volumes

So if you’ve looked at the requirements for your application and you decide that you need a volume larger than 2TB, what are your options with vSphere 5.x?

  1. Using one or more VMFS volumes with virtual disks up to 2TB and in guest volume managers to concatenate them. Implications: The more devices the more storage IO queues and potentially the more performance. Oracle RAC vMotion Supported. Theoretically supports up to 120TB storage per VM.
  2. Physical Mode RDM – Support up to 64TB individual device, more than 3PB per VM.Implications: No Storage vMotion, No Hypervisor Snapshot Support, No Cloning, No vSphere API’s for Data Protection Support (vADP), No vCloud Director Support, No FT Support, No Oracle RAC vMotion Support, No Clustering vMotion Support.
  3. In Guest iSCSI – Supports up to 16TB or greater individual devices depending on iSCSI target. Implications: No Storage vMotion (of iSCSI devices), No Hypervisor Snapshot Support (of iSCSI devices), No Cloning (of iSCSI devices), No vSphere API’s for Data Protection Support (vADP) (of iSCSI devices), vCloud Director Supported, FT Supported, vMotion Supported, Clustering vMotion Support, higher CPU utilization.
  4. In Guest NFS – Supports very large volumes depending on the array. Implications: No Storage vMotion (of NFS devices), No Hypervisor Snapshot Support (of NFS devices), No Cloning (of NFS devices), No vSphere API’s for Data Protection Support (vADP) (of NFS devices), vCloud Director Supported, FT Supported, vMotion Supported, Oracle RAC vMotion Support, higher CPU utilization.

You can’t evaluate the alternatives in isolation and to be fair they are workarounds that you wouldn’t even have to consider if larger than 2TB VMDK’s were possible. Physical Mode RDM’s in particular have operational implications, especially as you can’t use hypervisor snapshots, cloning, and no backup API integration, just to name a few. So any alternative you choose needs to be thoroughly considered.

The Gotcha with VMFS

If you are going to have databases or systems with large disk footprints (and have multiple per host) you may need to modify the ESXi VMFS Heap Size by changing the advanced setting VMFS3.MaxHeapSizeMB. Review KB 1004424 and Jason Boche’s article Monster VMs & ESX(i) Heap Size: Trouble In Storage Paradise. Currently VMFS5 is limited to a maximum of25TB of virtual disks open per host (Yes per host). With a default setting allowing only 8TB of VMDK’s to be open per host. This means even if it is acceptable to you for a single VM to have multiple virtual disks of 2TB and using in guest volume managers you would not be able to configure or open more than 25TB total maximum on a single host (was 32TB with VMFS3). This is why the limit of 120TB per VM on VMFS is at this point purely theoretical.

If you want to work around this limitation you will need to adopt option 2, 3 or 4 above or use virtual mode RDMs. The reason is this limit is purely with VMFS and doesn’t impact RDM’s (physical or virtual) or in guest iSCSI or NFS.

[Updated 20/09/2012] A great example where it would be good to be able to support > 25TB VMDK’s per host and > 2TB per VMDK is where a customer has a requirement such as virtualizing 20 x 4TB File Servers. Each fileserver may not need much in the way or RAM or CPU, but does need a decent amount of storage. In theory these 20 VM’s could easily be consolidated on a single host (although wouldn’t be for availability requirements), but because the VMFS limitation this is not possible, and due to the limit of 2TB per VMDK limit you will require a minimum of 2 VMDK’s per VM. It may be more convenient to have a single 4TB VMDK for these types of servers. One option is to design for a consolidation ration of 5:1 and size the physical hosts accordingly, making sure to increase the default VMFS heap size. However this would introduce additional operational costs and effort. This brings us back to option 2, 3 and 4 above again. In this case vRDM may be a better option than pRDM even with the 2TB limit as it allows easy migration to VMFS / VMDK’s in the future. pRDM would have the advantage of reducing the number of LUNs in total required for the VM’s, which might be 60 LUNs in total, not taking into account other VM’s and LUNs in the cluster (which could bring them close to the 256 LUN limit per host), but with a tradeoff of a harder migration path in the future.

Final Word

Microsoft appears to have put the cat squarely among the pigeons in terms of large virtual disk storage support with their latest release of Windows 2012 and Hyper-V. In this respect VMware is indeed playing catch up. But are greater than 2TB virtual disks really required right now for most applications? In my opinion no. For the majority of applications the existing vSphere hypervisor can adequately cater for their size and performance needs. But this is only going to last so long. There are some good use cases documented in Cormac Hogan’s blog article How Much Storage Can I Present to a Virtual Machine.

Most applications in my experience, especially the performance and latency sensitive messaging and OLTP database applications would benefit more from a greater number of SCSI devices and queues. In their case supporting more than 256 datastores per host would be of benefit, especially if there are multiple of them all grouped in a cluster.  The benefits of using VMFS and virtual disks are compelling and not being able to support very large virtual disks is definitely going to be a major problem in the future, considering VMFS5 already supports 64TB volumes. Especially considering the explosive growth of data. But do we want larger virtual disks and to sacrifice functionality, such as snapshots? I don’t think so. I hope that VMware will support larger virtual disks, even if they increase it up to 4TB or 16TB, and without sacrificing functionality. However in the meantime the alternatives such as RDMs and in guest storage access will fill the gap for some of the minority of workloads that need it, with the resulting trade offs in functionality. For those workloads where the workarounds are unacceptable they may not be virtualization candidates, at least on vSphere anyway, till some of these problems are solved.

Just because you can do something doesn’t mean you necessarily should. The back end array architecture needs to be considered and so does the data protection and disaster recovery protection aspects of the solution. It’s not good having a massive volume and a massive amount of storage per VM if you can’t protect that data and recovery it in a reasonable timeframe when required. I would like to know of your use cases that require greater than 2TB virtual disks and of your very large data Monster VM’s. Hopefully if there are enough customers that require larger than 2TB VMDK’s VMware will implement the necessary changes.

Here is what I’d like to see from VMware (In no particular order):

  • Larger than 2TB VMDK Support
  • More than 4 vSCSI Controllers per VM
  • More than 256 SCSI Devices per Host

I would be very interested to get your feedback on this.

Nghe nhạc cho vui

Ghi Chú

+ command "netstat -ano": IPs access

+ Restore dữ liệu trên SQLServer bằng dòng lệnh

RESTORE DATABASE db_name
FROM DISK = 'E:\db.BAK'
WITH REPLACE

+ Cover từ VB.NET sang C#

http://developerfusion.com/tools/convert/vb-to-csharp/

Certifications