Posted by: Anil | December 22, 2007

2008, going forward

Instead of reminiscing about how or what 2007 has been, I wanted to talk a little bit about where we are going with Entic Services (Entic.net). We are a pretty new shop, and the services we are currently offering (to the public) are only a small piece of the pie. Here is a sneak peak at whats coming in no particular order. We’re excited.

  1. New network provider coming soon. Our goal has been to provide quality bandwidth, and not quantity. This is exactly why you don’t see us offering 1500Gb bandwidth per month (or some large number) in our plans, which is usually a result of a server provider using poor quality network which is cheaper! We pay big bucks to get quality. More on this later.
  2. Your Apps™. This is kind of like Google Apps, but our version of it, with our own distinctions. Ours will be standards based services, unlike Google. More on this later.
  3. Expand our FreeABC.org and FreeXYZ.org (these domains don’t exist, its a secret, we can’t disclose them yet). More on this later.
  4. Better Billing. We are half developers, so we are tempted to write our own Billing system – this saves cost in the long term. However, we are still evaluating if Google Checkout or Paypal will work for us. Google might be a good choice if it had recurring payments, but it doesn’t yet. We’ll have to wait and see. More on this later.

Thats all for now. Our list isn’t complete, but more on that later.

🙂

Advertisements
Posted by: Anil | December 22, 2007

How to compile with Sun Studio

Sun Studio is Sun’s compiler and when used in building Solaris binaries, can increase performance of these applications. You can use some of the tips here to compile your own software on your Entic.net’s v.DS server.

Here is our environment before we start the build.

INSTALLDIR=/usr/local
LDFLAGS="-L$INSTALLDIR/lib -L/usr/sfw/lib -lsendfile -R/usr/sfw/lib"
PATH=/opt/SUNWspro/bin:/usr/ccs/bin:/usr/sbin:/usr/bin:
DMAKE_MODE=parallel
DMAKE_MAX_JOBS=8
CFLAGS="-fast -xipo -xtarget=opteron"
CC="/opt/SUNWspro/bin/cc"
CXX="/opt/SUNWspro/bin/CC"
export PATH CFLAGS DMAKE_MAX_JOBS DMAKE_MODE CC CXX INSTALLDIR LDFLAGS

Ignore some of those variables that seem out of place. Here’s how we’ve done a recent compile of Dovecot IMAP software after we’ve brought in those environment variables.

# make distclean; make clean
# ./configure --prefix=/usr/local/dovecot --with-ldap --without-pop3d --with-sqlite --without-gssapi --disable-ipv6 --without-docs
# dmake
# dmake install

dmake is the make that comes with Sun Studio. Using dmake is significantly faster at building than using GNU make, it runs the compile in parallel.
Thats it! The whole compile took less than two minutes to finish on our Quad core server. Let us know if you have problems compiling using Sun Studio, we can help!

Posted by: Anil | November 26, 2007

Backups and Restores

We are now officially backing up to USB devices, and no longer using tapes. Here are the new backup and restore policies in effect going forward:

Fri full

Friday at 3:30 am PDT, we kick off full backups to our USB drives. This data is always available from within Your server (e.g. this example server is vds5.entic.net):

root@vds5:/# ls /backup/vds5/
coolstack  fs         usrlocal
root@vds5:/#
File system Accessible from
/ /backup/vds5/fs/root
/opt/coolstack /backup/vds5/coolstack
/usr/local /backup/vds5/usrlocal

Linux/CentOS users: This online access to your backup data is only available to Solaris OS users. While your data is backed up the same way, the “/backup/vds5” is not accessible from within your server. Requests for restores need to goto support@entic.net.

1 hr snapshot

We also take 1 hour snapshots on top of the hour (e.g. 1:00, 2:00, 3:00, 4.00 PDT) of your system to the System hard disk (not to the USB device). If you deleted something in the last hour, we can restore it.

Unlike the friday full’s, we do not currently export these backups to Your server. To request a restore, you’ll need to email us or page us for immediate restore. We’ll provide some kind of script in the future to allow for self management of these 1 hr snapshot/restores.

Posted by: Anil | November 21, 2007

Coolstack and an apology

Entic.net hosts Coolstack software. Coolstack is a stack of software precompiled to run as fast as possible. Its optimized Solaris software.

Here is whats included as of right now (in the Entic.net implementation):

Lighttpd
PHP5
Perl 5.8.8
Tomcat 5.5.23
Apache
memcached
MySQL
Ruby

To make the Coolstack maintainable when multiple zones are used, we actually don’t do the Sun suggested implementation of installing through packages for each zone/server we provision. What we do is install the Coolstack packages in the global zone, and then using lofs, copy the /opt/coolstack bits into the customer’s ZFS partition. Here is a zonecfg snippet:

add fs
set dir=/opt/coolstack
set special=/data/vds5/coolstack
set type=lofs
add options rw
add options nodevices
add options exec
end

This has a few of big benefits:

  • If we remove or update packages of Coolstack in the global zone, it won’t effect our customer’s /opt/coolstack binaries
  • But, if we do need to roll out new versions of Coolstack software, we just update the global zone, and then do a rsync (w/o the delete option) to the customer’s partition
  • This also gives the customer full control over their server. Users are free to remove /opt/coolstack/* to get a blank system without Coolstack software

Now for the apology. We try really hard not to mingle in your server directly (logging in, change things on your live/running system, removing or adding packages) but sometimes this does happen.

We had to login to several of the zones earlier this month to resolve some issues with the above Coolstack implementation. We have it all ironed out now and you should no longer see us on Your server.

🙂

Posted by: Anil | November 21, 2007

Switching Databases From MySQL to PostgreSQL 8.2

Here is an excellent article on how to switch from MySQL to PostgreSQL. The steps outlined there apply to Entic.net v.Dedicated Servers.

The PostgreSQL RBAC role is actually included in the “vds” user account that is created when the v.Dedicated Server is created, as shown here:

vds19@vds19:~% profiles -l

      Postgres Administration:
          /usr/postgres/8.2/bin/initdb    uid=postgres
          /usr/postgres/8.2/bin/ipcclean    uid=postgres
          /usr/postgres/8.2/bin/pg_controldata    uid=postgres
          /usr/postgres/8.2/bin/pg_ctl    uid=postgres
          /usr/postgres/8.2/bin/pg_resetxlog    uid=postgres
          /usr/postgres/8.2/bin/postgres    uid=postgres
          /usr/postgres/8.2/bin/postmaster    uid=postgres
      All:
          *
vds19@vds19:~% profiles
Postgres Administration
Basic Solaris User
All
vds19@vds19:~%
Posted by: Anil | October 29, 2007

Glassfish Hosting on Solaris

[Update April 9th: This blog entry has been moved to HOWTO_Glassfish.]

In this blog, we’ll show how to setup Glassfish V2 b58g in your v.DS server from Entic.net. This server will be run as root, and listen on port 80 (http) and 443 (https). It’ll also be integrated into Solaris’ SMF, so it can come up on its own at boot.

First, become root. We’ll grab the Glassfish Solaris binaries for x86, from here using wget.

vds13@vds13:~% su -
Password:
Sun Microsystems Inc.   SunOS 5.10      Generic January 2005
root@vds13:/#
root@vds13:/# cd /usr/local
root@vds13:/# /opt/csw/bin/wget http://java.net/download/javaee5/v2_branch/promoted/SunOS_X86/glassfish-installer-v2-b58g.jar
...
root@vds13:/usr/local# ls
bin                              lighttpd
etc                              lighttpd-1.4
glassfish-installer-v2-b58g.jar  man
include                          sbin
lib                              share
root@vds13:/usr/local# java -Xmx256m -jar glassfish-installer-v2-b58g.jar
...
root@vds13:/usr/local# cd glassfish
root@vds13:/usr/local/glassfish# chmod -R +x lib/ant/bin
root@vds13:/usr/local/glassfish#

Glassfish binaries are installed in /usr/local/glassfish. Now, edit the setup.xml file. Change the default https.port (from 8181 to 443), and instance.port (from 8080 to 80) properties.

root@vds13:/usr/local/glassfish# vi setup.xml
...

Install the domain. We’ll use the default of domain name of domain1. This domain will be created in /usr/local/glassfish/domains/domain1.

root@vds13:/usr/local/glassfish# chmod 400 setup.xml
root@vds13:/usr/local/glassfish# lib/ant/bin/ant -f setup.xml
...

This version of Glassfish requires the password file for SMF integration. So, here it is. The passwords can be as shown, they are not used.

root@vds13:/usr/local/glassfish# cat > /usr/local/glassfish/passfile
AS_ADMIN_USER=admin
AS_ADMIN_PASSWORD=adminadmin
AS_ADMIN_MASTERPASSWORD=changeit
^D
root@vds13:/usr/local/glassfish# chmod 400 passfile; cat passfile
AS_ADMIN_USER=admin
AS_ADMIN_PASSWORD=adminadmin
AS_ADMIN_MASTERPASSWORD=changeit
root@vds13:/usr/local/glassfish#

Now we create the actual SMF file.

root@vds13:/usr/local/glassfish# bin/asadmin create-service --passwordfile /usr/local/glassfish/passfile --type das --serviceproperties net_privaddr /usr/local/glassfish/domains/domain1
The Service was created successfully. Here are the details:
Name of the service:application/SUNWappserver/domain1
Type of the service:Domain
Configuration location of the service:/usr/local/glassfish/domains
Manifest file location on the system:/var/svc/manifest/application/SUNWappserver/domain1_usr_local_glassfish_domains/Domain-service-smf.xml.
The service could be enabled using svcadm command.
Command create-service executed successfully.

Now to start it up.

root@vds13:/usr/local/glassfish# svcs *SUNWappserver*
STATE          STIME    FMRI
disabled       18:36:09 svc:/application/SUNWappserver/domain1:default
root@vds13:/usr/local/glassfish# svcadm enable svc:/application/SUNWappserver/domain1:default
root@vds13:/usr/local/glassfish# svcs *SUNWappserver*
STATE          STIME    FMRI
online         18:42:21 svc:/application/SUNWappserver/domain1:default
root@vds13:/usr/local/glassfish#

Thats it… and its online!

You can now access the application at http://vds13.entic.net (or https://vds13.entic.net). The admin console can be accessed at http://vds13.entic.net:4848.

Now, lets finalize things and go change the admin password.

root@vds13:/usr/local/glassfish# bin/asadmin change-admin-password Please enter the old admin password>
Please enter the new admin password>
Please enter the new admin password again>
Updated .asadminpass file with new password.
Command change-admin-password executed successfully.
root@vds13:/usr/local/glassfish#
Posted by: Anil | October 16, 2007

Distributed Speed Test & Monitoring

The problem
How fast is one network or server over another? There are lots of sites out there now that do speed tests to showcase a network operator’s speed. These speed tests (using flash enabled speedometers) attempt to show how fast data can be uploaded and downloaded to the user, from the server or website. However, these speed tests are meaningless and providing nothing more than a little entertainment.

The fact is, there are a lot of variables that contribute to how fast a site is to the perspective website visitor. The visitor’s computer performance, DSL or Cable usage in the neighborhood, an Internet Provider’s network capacity, the time of day, number of people visiting the same site at the same time, etc. The point here is, all of these factors impact how long it takes to download that 1Gb file from that web site. So, how can a single visitor looking at these speedometers really know how good a network is? You can’t! The network being speed tested is different for different people.

The solution
This is where we developed the Distributed Network Speed Test and Monitoring, which we call Speed Test and Monitoring 2.0 Similar to SETI@Home, the Distributed Network consists of people all over the world who run a small Java client that basically just sits in the background and collects speed/performance metrics to sites in our database.

For example, the site www.entic.net is pinged from all of the people running this Java client. The aggregate data is collected and then published on a Google map mashups. Adding these Hosts to the basket, and then comparing them, will show comparison Reports that help to determine how fast and how reliable a network/site is compared to another.

Similar in concept to sites like just-ping.com, we believe our Distributed Networking concept is better. The data collection nodes on sites like these are sitting in a data center somewhere with low latencies, which don’t accurately demonstrate a site’s performance. In comparison, Our Distributed Network comprises of users running Java clients on their PC at home, which is based on more factual speed tests.

The same concept used to determine speed tests in our Distributed Network is also used for doing distributed monitoring with better accuracy rates from real users across the globe. More on the monitoring piece in another blog. This one is more geared for speed tests…

On one of our v.Dedicated servers, we compared the benchmarks generated with UnixBench when compiled with the default GNU GCC (which is what almost 99% of all hosting/server providers use) vs Entic optimized Solaris native compiler. No tuning was done to UnixBench itself.

Here are the results for GCC run:

                     INDEX VALUES
TEST                                        BASELINE     RESULT      INDEX

Dhrystone 2 using register variables        116700.0  4455048.7      381.8
Double-Precision Whetstone                      55.0      887.5      161.4
Execl Throughput                                43.0      626.2      145.6
File Copy 1024 bufsize 2000 maxblocks         3960.0    49992.0      126.2
File Copy 256 bufsize 500 maxblocks           1655.0    12987.0       78.5
File Copy 4096 bufsize 8000 maxblocks         5800.0   158381.0      273.1
Pipe Throughput                              12440.0   323922.0      260.4
Pipe-based Context Switching                  4000.0    39441.5       98.6
Process Creation                               126.0     1144.9       90.9
Shell Scripts (8 concurrent)                     6.0      164.0      273.3
System Call Overhead                         15000.0   264727.8      176.5
                                                                 =========
     FINAL SCORE                                                     166.5

and (drum rolls please…) for Entic’s optimized results:

                     INDEX VALUES
TEST                                        BASELINE     RESULT      INDEX

Dhrystone 2 using register variables        116700.0  7853579.1      673.0
Double-Precision Whetstone                      55.0     1621.8      294.9
Execl Throughput                                43.0      737.1      171.4
File Copy 1024 bufsize 2000 maxblocks         3960.0    80697.0      203.8
File Copy 256 bufsize 500 maxblocks           1655.0    32700.0      197.6
File Copy 4096 bufsize 8000 maxblocks         5800.0   235523.0      406.1
Pipe Throughput                              12440.0   282546.0      227.1
Pipe-based Context Switching                  4000.0    37091.1       92.7
Process Creation                               126.0     1210.4       96.1
Shell Scripts (8 concurrent)                     6.0      165.0      275.0
System Call Overhead                         15000.0   162554.1      108.4
                                                                 =========
     FINAL SCORE                                                     209.2

Thats more than 20% performance improvement!

The servers you purchase from Entic are all optimized to run the fastest. You’ll be sure to get the fastest performance possible for your web serving.

Posted by: Anil | April 9, 2007

Getting started guide

THIS PAGE HAS BEEN MOVED TO WIKI.ENTIC.NET!

THIS PAGE HAS BEEN MOVED TO WIKI.ENTIC.NET!

THIS PAGE HAS BEEN MOVED TO WIKI.ENTIC.NET!

THIS PAGE HAS BEEN MOVED TO WIKI.ENTIC.NET!

Table of contents

Solaris Beginner’s Guide

SMF

SMF is a framework for managing the services that run on your Solaris server. To get started you only need to know two commands: svcs and svcadm. Running just “svcs -a” will show all the services that are either enabled or disabled.

root@vds3:/# svcs ftp
STATE          STIME    FMRI
online         11:20:48 svc:/network/ftp:default
root@vds3:/#

That command shows FTP is running. To disable FTP to your server, you would do:

root@vds3:/# svcadm disable ftp
root@vds3:/# svcs ftp
STATE          STIME    FMRI
disabled       11:43:56 svc:/network/ftp:default
root@vds3:/#

Filesystem layout
When you first login to your server, the filesystem looks confusing. Don’t panic. This is because of how your server inherits parts of the filesystem from the global OS. This is a good thing, this allows Your Server to be managed by us, with ease.

root@vds3:/# df -h
Filesystem             size   used  avail capacity  Mounted on
/                      5.0G  1010M   4.0G    20%    /
/dev                   5.0G  1010M   4.0G    20%    /dev
/lib                    13G   5.8G   6.6G    47%    /lib
/opt/SUNWspro           13G   5.8G   6.6G    47%    /opt/SUNWspro
/opt/coolstack         5.0G  1010M   4.0G    20%    /opt/coolstack
/opt/sfw                13G   5.8G   6.6G    47%    /opt/sfw
/platform               13G   5.8G   6.6G    47%    /platform
/sbin                   13G   5.8G   6.6G    47%    /sbin
/usr                    13G   5.8G   6.6G    47%    /usr
/usr/local             5.0G  1010M   4.0G    20%    /usr/local
proc                     0K     0K     0K     0%    /proc
ctfs                     0K     0K     0K     0%    /system/contract
mnttab                   0K     0K     0K     0%    /etc/mnttab
objfs                    0K     0K     0K     0%    /system/object
swap                    10G   304K    10G     1%    /etc/svc/volatile
/usr/lib/libc/libc_hwcap1.so.1
                        13G   5.8G   6.6G    47%    /lib/libc.so.1
fd                       0K     0K     0K     0%    /dev/fd
swap                    10G     4K    10G     1%    /tmp
swap                    10G    12K    10G     1%    /var/run

Some of the filesystems like /lib, /sbin, /usr, are inherited files from the parent OS. These file systems are read-only. If we ever update something like /bin/bash with a new patch, Your Server will automatically get this updated because it is inherited. The benefit is that You don’t have to manage core Solaris software, we’ll do it for you!

At the same time, you of course have full write access to /, /etc, /tmp, /opt, and /usr/local where you can make any customizations to Your Server.

HTTP Server (with PHP support)

Your server has two versions of HTTP servers installed, Apache2 and Lighttpd. Both of them are based on Solaris Cool Stack. We recommend using these instead of building your own. The DocumentRoot for Lighttpd is: /opt/coolstack/lighttpd/htdocs. For CoolStack Apache2, its: /opt/coolstack/apache2/htdocs. This is where you’ll need to put your web pages.

To enable CoolStack lighttpd:

root@vds3:/# svcadm enable CSKlighttpd
root@vds3:/# svcs *lighttpd*
STATE          STIME    FMRI
online          7:28:49 svc:/network/csk-lighttpd:CSKlighttpd
root@vds3:/#

Now you can goto http://vds3.entic.net, and you should see the lighttpd serving pages. You can also visit http://vds3.entic.net/phpinfo.php for PHP information.

You can follow similar steps for Apache2. Remember, the DocumenRoot for Apache2 is in a different place for Lighttpd.

Databases

We currently provide two databases, PostgreSQL and MySQL that are integrated with SMF. The binaries for sqlite3 also exist in /usr/local/bin, if needed.

PostgreSQL is the native Solaris database. It is disabled by default. Your Server actually comes with PostgreSQL 8.1 and 8.2. 8.2 is the preferred database and is shown here. Your “admin” user account is configured to have full access to the PostgreSQL database. You can manage your PostgreSQL server using that account, root access is not needed.

Initialize

Before you can use PostgreSQL, you need to initalize it. The following command is run as root.

root@vds3:/# su - admin -c "pfexec /usr/postgres/8.2/bin/initdb /var/postgres/8.2/data" ...

The database data files are in /var/postgres/8.2 and the configuration file is /var/postgres/8.2/data/postgresql.conf. You are all set now!

Starting PostgreSQL

To enable PostgreSQL, login to your “admin” account and then run:

admin@vds5:~% svcs *post*
STATE          STIME    FMRI
disabled       20:20:51 svc:/application/database/postgresql:version_81
disabled       20:53:23 svc:/application/database/postgresql:version_82
admin@vds5:~% /usr/sbin/svcadm enable svc:/application/database/postgresql:version_82

admin@vds5:~% svcs *post*
STATE          STIME    FMRI
disabled       20:20:51 svc:/application/database/postgresql:version_81
online         20:54:48 svc:/application/database/postgresql:version_82
admin@vds5:~%

Your “admin” account is actually setup with a role of the postgres user which allows for PostgreSQL database management without being root.


MySQL
is built with the Solaris CoolStack. The binaries are in /opt/coolstack/mysql_32bit/bin (You should set your PATH environment variable to search this directory first).

Initialize

root@vds3:/# groupadd mysql
root@vds3:/# useradd -c "MySQL Reserved UID" -d / -g mysql -s /usr/bin/false mysql
root@vds3:/# /opt/coolstack/mysql_32bit/bin/mysql_install_db
root@vds3:/# chown -R mysql:mysql /opt/coolstack/mysql_32bit/data
root@vds3:/# chmod -R 770 /opt/coolstack/mysql_32bit/data

root@vds3:/# /opt/coolstack/mysql_32bit/bin/mysqld_safe &
root@vds3:/# /opt/coolstack/mysql_32bit/bin/mysqladmin -u root password 'mydefaultpassword'
root@vds3:/# /opt/coolstack/mysql_32bit/share/mysql/mysql.server stop

The above commands setup mysql for the first time. You only need to do that once. Please be sure to pick your own password, replace mydefaultpassword with your own password.

The database data files are in /opt/coolstack/mysql_32bit/data.

Starting MySQL

root@vds3:/# svcs *mysql*
STATE          STIME    FMRI
disabled        9:34:39 svc:/network/csk-mysql32:default
root@vds3:/# svcadm enable svc:/network/csk-mysql32:default
root@vds3:/# svcs svc:/network/csk-mysql32:default
STATE          STIME    FMRI
online          9:35:15 svc:/network/csk-mysql32:default
root@vds3:/#

Webmin

For new users to Unix, you might want to use Webmin. Webmin allows web based management of your server. Webmin is installed by default, but is not configured.

root@vds3:/# cd /usr/local/webmin
root@vds3:/# ./setup.sh
***********************************************************************
*            Welcome to the Webmin setup script, version 1.400        *
***********************************************************************
Webmin is a web-based interface that allows Unix-like operating
systems and common Unix services to be easily administered.

Installing Webmin in /usr/local/webmin-1.400 ...

***********************************************************************
Webmin uses separate directories for configuration files and log files.
Unless you want to run multiple versions of Webmin at the same time
you can just accept the defaults.

Config file directory [/etc/webmin]: /usr/local/etc/webmin
...

When asked to enter Config file directory, be sure to type in /usr/local/etc/webmin. Be sure to read the last output of that command, it’ll tell you how to connect to webmin on your server.

Remove Customizations

Your server actually comes with minimal customizations done by Entic.net, otherwise it is standard Solaris 10 u4 build. For those power users who don’t want these minimal customizations, here is how to get your server back to the native Solaris OS.

Remove local software:

root@vds19:/# rm -rf /usr/local/*
root@vds19:/#

Remove Coolstack and its integration with SMF. Disable services that might be online:

root@vds19:/# svcs *csk*
STATE          STIME    FMRI
disabled       Nov_14   svc:/application/csk-tomcat:CSKtomcat
disabled       Nov_14   svc:/network/csk-lighttpd:CSKlighttpd
online         Nov_14   svc:/network/csk-mysql32:default
online         Nov_17   svc:/network/csk-http:CSKapache2
root@vds19:/# svcadm disable svc:/network/csk-mysql32:default
root@vds19:/# svcadm disable svc:/network/csk-http:CSKapache2
root@vds19:/# svccfg delete svc:/network/csk-http:CSKapache2
root@vds19:/# svccfg delete svc:/network/csk-lighttpd:CSKlighttpd
root@vds19:/# svccfg delete svc:/application/csk-tomcat:CSKtomcat
root@vds19:/# svccfg delete svc:/network/csk-mysql32:default
root@vds19:/# rm -rf /opt/coolstack/*

That should do it!

Java

Our Solaris 10 build comes with JDK 1.4, 1.5 and 1.6. These versions of all available in /usr/jdk. We recommend using Java from /usr/jdk/latest, which always points to the latest version of Java available on the system. If we upgrade to newer Java, the /usr/jdk/latest will updated to point to upgraded Java.

« Newer Posts

Categories