March 17th, 2009 
Kiwi Si 
Until recently I had been running my ESX VM’s on local disk. This is mostly due to not having enough time to get some shared storage up and running.
I however was determined to get something up and running for my ESX lab so that I can play around with some of ESX’s more powerful, and interesting, features such as DRS, HA and VMotion.
As with most of you money is a serious consideration so as I am not in a position to implement a fibre attached SAN solution – though this would be nice. The next best option is iSCSI. I am running both VMware ESX 3.5 and ESXi 3.5 in my lab and both provide iSCSI functionality by default to connect through to an iSCSI target.
There are a handful of good free (free is always good 
) iSCSI software that can be downloaded. Some are standalone installs, others come in the form of virtual appliances and some both.
Here is a list of those that I know of (there will no doubt be many more):
- OpenFiler
- Free NAS
- Xtravirt
- EMC Celera (VM) – log into the EMC portal and search for ‘Celerra Simulator’.
I decided to give OpenFiler a go – as I’d heard good things about the latest release, v2.3. Here’s a link to a really good document on the OpenFiler site that details the underlying architecture.
The download link to OpenFiler can be found here.
As you can see there are many different downloads available. I decided to install it on old Shuttle XPC PC I had lying about. This server has 1GB of memory, Single AMD64 2GHz CPU and a single Gigabit NIC all of which are more than adequate for running OpenFiler. I downloaded and installed the x86_64 version.
I then burnt the ISO to CD and installed OpenFiler using the great graphical step by step installation guide provided on their web site.
For this lab install I have used the following network configuration.
OpenFiler (Shuttle PC)
NIC: Single port 1Gb
IP: 10.0.0.1/24
ESX Server (ML110 G5)
ESXi 3.5.0 U3
NIC: Dual port NIC
IP#1 (iSCSI): 10.0.0.10/24
IP#2 (General Traffic): 192.168.1.11/24
Management Server (Home Brew PC)
OS: Windows 7
NIC: Dual port NIC
IP#1 (iSCSI network – for management of OpenFiler): 10.0.0.2/24, IP#2 (General Traffic): 192.168.1.1/24
Once you have OpenFiler installed you can then access a web based management console which allows you to configure your new OpenFiler installation.
Opening a web browser and pointing it to the IP address (ie: https://<IP of OpenFiler>:446> of the OpenFiler server you should be presented with a logon screen like that below (love the fat Linux penguin).
At the prompt enter ‘openfiler’ for the ‘Username’ and ‘password’ for the password. These details can be changed once you’ve successfully logged onto the management portal along with the ability to create additional accounts and groups.
Step 1 – Network Access Configuration:
The first thing to do is set up the ‘Network Access Configuration’. This is the host or subnet (depending on how granular you want the access to be) you wish to provide access from. Select the ‘System’ tab and from the ‘Network Access Configuration’ section at the bottom of the page enter in either the IP from which you wish to access the OpenFiler from or enter in a whole subnet from which the Open filer will accept traffic from.
As I am running a secure lab environment I am just going to enter in the whole subnet for ease. You may want to be a little more granular if using this in a production or non-secure environment.
Make sure that the ‘Type’ is set to ‘Share’.
Volumes – Important Information (for the clarity of mind).
Before we go any further I just want to point out some potential confusion that can arise when creating, assigning and configuring the various ‘Volume’ parameters within OpenFiler. There are 3 volume items we’ll be dealing with in the next few steps. I personally found this a little confusing at first so thought I’d try and throw down some clarification around this area:
1. Physical Volume – Assigning space on a physical disk for use in a Volume Group.
2. Volume Group – Contains Physical Volumes from which a Logical Volume will be created.
3. Logical Volume (LUN) – This is what is presented through to a server (eg: ESX).
Now with that out of the way.. on with the show!
Follow these steps in order to configure your OpenFiler SAN and present it through to VMware ESX/ESXi.
Step 2 – Create a New Physical Volume:
We need to create a physical volume which we will then present through to a Volume Group. To do this select ‘Block Devices’ from the ‘Volumes section’ menu.
Select ‘Edit Disk’ on the hard disk you want to create this new physical volume.
Scroll to the bottom of the screen and you will see the available spare space on this disk along with some other parameters. If you are not intending to create a RAID set for your physical volume then select ‘Physical volume’ as your partition type and select the ‘Mode’ as ‘Primary’.
Adjust the start and end cylinders to determine the size of physical volume and when satisfied press the ‘Create’ button.
You will then be shown a summary of the partitions on this disk. Notice that the ‘Physical Volume’ I just created appears on the list (bottom).
Step 3 – Create a New Volume Group:
Next we want to create a new ‘Volume Group’ for the ‘Physical Volume’ we created to reside in. Click on the ‘Volumes’ tab and then select ‘Volume Groups’ from the ‘Volume section’ menu on the right hand side menu.
Enter in a ‘Volume group name’ and select (check box) the physical volume to which you wish to associate the Volume Group. Then press ‘Add Volume group’
You should now be presented with a new Volume Group that looks like this:
Step 4 – Create a Volume:
We now want to create a ‘Volume’. Click on ‘Add volume’ from the right hand ‘Volumes section’ menu.
Now select the ‘Volume Group’ you just created and press the ‘Change’ button. You will now be presented with the following screen where you determine the size of the ‘Volume’ your going to create within your ‘Volume Group’.
For this example I’m going to create a ‘Volume’ that occupies the entire space of the ‘Volume Group’. Enter in the ‘Volume Name’ and determine the size by either keying in the required space or using the slider bar. Then for the ‘Filesystem/Volume type’ select ‘iSCSI’. This lasts part is important to all of this working so make sure it is set correctly (ie: iSCSI)!
When your happy with your settings press the ‘Create’ button.
Once the ‘Volume’ is created you will be greeted with a screen with a nice big green coloured pie chart in it which is indicating the amount of the ‘Volume Group’ that the volume has consumed. Which in this example is all of it.
This is now everything to do with creating a volume completed. We now want to enable the connectivity side of things (ie: allowing other PCs/Servers to connect to the OpenFiler SAN).
Step 5 – Enable the iSCSI Target Service:
Click on the ‘Services’ tab of the main window.
Next click on the ‘Services’ tab and enable the ‘iSCSI target server’ (see below). By default it is set to ‘Disabled’. For connecting the OpenFiler SAN through to an VMware ESX/ESXi host we don’t need any of the other services enabling.
Step 6 – Add an iSCSI Target:
Returning to the ‘Volumes’ section of the OpenFiler web interface select ‘iSCSI Targets’ from the ‘Volumes section’ menu on the right hand side of the screen.
We first want to create a new iSCSI target and do by select the first sub-tab called ‘Target Configuration’ in the ‘iSCSI Targets’ section. I personally keep the default ‘Target IQN’ generated by OpenFiler though you can alter it at this stage if your want. Now press the ‘Add’ button.
You will now be given a screen with a summary of the settings for the new iSCSI Target.
Step 7 – Map the LUN:
Now select the ‘LUN Mapping’ tab and click on the ‘Map’ button. There are no other settings that need changing.
Step 8 – Allow access to the iSCSI Target:
Select the ‘Network ACL’ tab and from the ‘Access’ drop down list box select ‘Allow’ (Note: it is set to ‘Deny’ by default) and then press the ‘Update’ button. This allows the IP range we defined in step 1 access to the iSCSI Target we just created.
You may have noticed the next menu tab which is called ‘CHAP Authentication’. In this section you would specify a logon name and password with incoming access to this iSCSI target. I am not worrying about configuring this as it is just a temporary set up for my test lab. Though if you are think about setting something up which will be a little more permanent then I’d definitely recommend enabling CHAP authentication. This’ll need enabling and these credentials specifying on the ESX side of things – but is very easy to do.
Open Filer Configuration Stage Finished!
This is now a basic OpenFiler configuration up and running with a LUN ready to be added to ESX.
IMPORTANT NOTE: If you are ever thinking of implementing OpenFiler for use with ESX in a production environment then it is highly recommended to keep the iSCSI network separate (for security and performance) from all other general type traffic and to apply CHAP’s encryption. In my lab I have created two VLAN’s on my Linksys SLM2008. One for general network traffic and the other for the iSCSI traffic.
Step 9 – VMware ESX iSCSI Configuration:
Open up the VMware Virtual Infrastructure Client (VIC) and select the ESX server that you want to add the iSCSI storage to. From the ‘Configurations’ tab in the right pane select ‘Networking’. You will now be presented with the networking configuration for that ESX server.
As you can see my particular install of ESX is using ESXi.
I have created a second Virtual Switch and have allocated a VMKernel port on the second NIC port (which is patched into the 10.0.0.0/24 VLAN) which I have given a 10.0.0.11/24 IP address. The is the port that the iSCSI traffic will use.
If you are configuring this using ESX 3.5 then you will also have to add a ‘Service Console’ port to this newly created Virtual Switch. If you don’t do this you’ll get this friendly reminder:
We now want to look at the ‘Storage Adapters’. Click on this option in the ‘Hardware’ menu.
You will see that there is an iSCSI Software Adapter already in place – though currently not enabled. All that needs doing is to configure and point it to the OpenFiler LUN(s). Notice how all the iSCSI related details are currently blank.
Click on the ‘iSCSI Software Adapter’ and select ‘Properties’.
The status of the iSCSI Software Adapter is initially set to ‘Disabled’. We want to enable it and assign the relevant details. Click on ‘Configure’. Check the ‘Enable’ status box and click ‘Ok’
The iSCSI properties will now be populated (see below). I won’t go into the format of the iSCSI name and alias though the VMware Infrastructure 3: Advanced Technical Design Guide and Advanced Operations Guide has a good section on iSCSI and explains these details.
Clicking on the ‘Dynamic Discovery’ tab and selecting ‘Add’ we are presented with an ‘Add Send Targets Server’ dialogue box. This is where we’ll enter the IP address of our OpenFiler server/SAN. (Note: the ‘Static Discovery’ tab is only used when using a hardware iSCSI initiator)
After entering in this IP information and pressing ‘Ok’ it can take a little while before iSCSI server is detected.
With these new iSCSI setting we are prompted to re-scan the host. Select ‘Yes’
After ESX has finished its re-scan you should now see the LUN(s) appear that you created in OpenFiler.
With this shared storage and a couple of ESX servers you can now start using some of the more interesting and powerful features of VMware ESX such as VMotion and HA (assuming you have the appropriate license ).
As mentioned this is purely intended as a rough guide to getting OpenFiler up and running with VMware ESX. OpenFiler has plenty of other great features and is worth investing some time into.
Hope this guide helps. Good Luck!
Related posts:
- VMware ESX(i) 4.0 (vSphere) -Connecting to an iSCSI Storage Target. With a complete re-write of the iSCSI portion of...
- Link – Good Tips for using iSCSI with VMware ESX Not sure where the last week has gone. Things...
- VMware vSphere ESX: Install, Configure, Manage – Preparing your Test Lab I’m sure many of you that are fortunate enough...
- London VMware Users Group (VMUG) – November 2009 Just a quick post to remind those of you...
- What is an HP LeftHand iSCSI SAN? There have been some great IT technologies and products...
Posted in 
My name is Simon Seagrave and I am a London (UK) based Technical Architect. 
Loading ...
Nice post but that’s a lot of work!
I use my general purpose Solaris mail/file server as my ESX iSCSI storage:
zpool create mirror c0d0 c1d0 mypool
zfs create -o shareiscsi=on -V 250TB mypool/vmstorage
..that’s all that’s required on the server side and I get all the advantages of ZFS too.
Peter
[Reply]
Peter, sure, but anytime you use a cli vs a gui the APPEARANCE is shorter. But really, however you setup the iSCSI piece, you still have to setup vmware iSCSI and that is over half this article.
[Reply]
Hi
You show only ONE ESX server in your configuration. Have you tested with 2?
Because Openfiler authorize only one iSCSI connection at a time… How do you do to connect the two ESX and test HA, Vmotion and so one?
Regards,
Ced.
[Reply]
Late reply, better than never
I just finished implementing 3 ESX servers targeting the same 2 LUNs provided by an OpenFiler 2.3 server. I’m here to tell you that vMotion works flawlessly, watching the status page of OF I don’t see any reason for concern with system resources.
Pre-requisite: Virtual Center, I’m using 2.5.
[Reply]
thank you very muchhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh
[Reply]
I a noob to VMware, so bare with me. Im trying to build lab VMware for my house, I’m stuck at the networking part where you mentioned you’ve created a second virtual switch with VLAN. I just have a regular hub and Im unable to perform a VLAN. Am I required to perform a VLAN? Plus, I dont see the “ISCSI software adapter” option, under storage adapters.
VMware ESX 3.5
[Reply]
I have updated the security profile / firewall for the software iscsi client and still cant see the iscsi software adapter under storage adapters… am i doing something wrong
[Reply]
Very appreciated. By following your guides, I am able to create iscsi to share data between 2 esxi servers.
[Reply]
I’ve followed the instructions and can make the connection but show 0 targets.
Any ideas? I’m using ESXi 4.5.0 and openfiler 2.3
Have checked the LUN mapping and network ACL access on openfiler but no luck.
The iscsi connection is on a separate NIC and switch in ESXI.
[Reply]
I’have a few questions – probably someone can help me in this (esx 3.5U2 and openfile 2.3).
The iqn entered in the esx server (name change requires a reboot) must be the inititor name (and therefore different from the iqn of the iqn target name as defined in the openfiler – is this right ?
I have 2 esx-servers – for failover they should connect to the same storage. Can they connect to the same iqn or is it better to provide a secondary target iqn for the second esx ?
After some experiments with bonded/trunked ports it looks like my share was damaged (coud not reconnect). Are there any tools to check vmfs storages or to
[Reply]
The only thing that took a lot of work was the time Simon put into making this post so the rest of us would spend LESS time getting it up and running. Thank Simon!
[Reply]
I have two VI ESX 3.5 boxes each with 2 NICs on an isolated 192 subnet and an Openfiler 2.3 with 2 NICs on the same subnet. How would I configure HA without a DNS server? Thx!
[Reply]
Hi Al,
I’m not sure I 100% understand the question. Are you talking about HA on ESX or OpenFiler?
Cheers,
Simon
[Reply]
Hi my name is Donald. i am trying to configure openfiler iscsi for my VMware ESX Server if you can help i would appreciate it very mach.
Thank you
Donald
[Reply]
Hi Donald,
What problem(s) are you experiencing?
Cheers,
Si
[Reply]
Hoping someone can help me test OF 2.3 and esxi iscsi connection. Steps are below:
1. Installed OF 2.3 as a vm on esxi (host#1). (Did not use appliance)
2. configured iScsi on OF and was able to map the lun.
3. On a separate esxi (host#2), I’m trying to connect to the iScsi target.
a. On host#2 I created a iscsi connection using vmkernel on a seperate port from the esxi host management network
b. My ip address for the iscsi connection is in the same subnet as my OF vm.
c. On the iSCSI Software Adapter (vmhba32) of host#2, I’ve enabled and configured the iScsi initiator
d. On OF, the IP of the step(a) is allowed access to the target.
questions:
1. No luns appear after scan job. What happen? what have I missed?
[Reply]
I followed your guide and I have created an iscsi target in OF 2.3 and I mapped the LUN and granted access to the ESX server. I configured the iscsi storage adapter in VI but I cannot get it to see the storage any help or advise out there?
[Reply]
In my case I could not get the iscsi storage to show up until I corrected the values under “network access configuration”
for my network the values are:
name: localnet
network/host: 10.1.10.0
netmask: 255.255.255.0
type: share
[Reply]
OF has an ip of 192.168.0.155
my ESXi box has an ip of 192.168.0.127
I added the ip for my esx box in OF and also added the ip access in the Network ACL list but I try to add the ip for OF in Virtual Infrastructure and it scans but does not find any available storage. I only have one nic so I cannot create a virtual switch for iscsi.
I have both connected directly to an 8 port gigabit switch and my OF box has two nic’s that are teamed to the ip listed above.
Any ideas on how to get this working?
I am running ESXi off of a USB Flash drive but everything seems to be working I can get into Unsupported mode on ESX and it sees everything correctly it is just not seeing the iscsi storage on OF.
[Reply]
Did you have to open up port 3260 on either OF or your ESX server or both? just wondering if this could be my issue. I have a D-link 8 port gigabit switch that both my ESX and OF are plugged into and the switch is plugged into a D-link wireless N router.
[Reply]
Awesome instructions. I was able to hook up my lab in one afternoon. I am also getting more involved in ESX, Sharepoint, Windows Server 2003K, 2008, and SBS.
Thanks for such good references. I will add you to my favorites.
[Reply]
Hi Sam,
Thanks for the feed back. I have a ton of ideas for future articles so keep checking back.
Cheers,
Si
[Reply]
Hi CDub,
Are you still having problems with this?
[Reply]
I figured out the problem in Open Filer I had to set my subnet mask for my ESX box to 255.255.255.255 and then it will see the iSCSI shares in ESX if I set the subnet mask to 255.255.255.0 it will not see the iSCSI volume in ESX. Hope this helps others out there trying to set this up on a 192.168.x.x subnet.
[Reply]
[...] HOW TO CONFIGURE OPEN FILER iSCSI to ESXi Server [...]
I also found that the iSCSI adapter in VMWare wouldn’t detect the LUN for some reason so I also had to edit the file on OpenFiler /etc/initiators.deny and remark the line defining the target. (using vim from command line)
Not sure what implications this has but it works for my test environment.
[Reply]
Hi Peter,
Thanks for taking to time to let us know – that will no doubt help others.
Cheers,
Si
[Reply]
This looks great! I tried to connect 2 ESXi servers to the same LUN on my iSCSI SAN (Buffalo Terastation Pro II iSCSI), and each time that I connect the ESXi server, it wipes out the storage on the LUN, and I cannot connect with the 1st ESXi anymore.
I have only 1 question: How do you connect 2 ESXi servers to the same iSCSI LUN simultaneously?
(I’d like to use VMotion)
[Reply]
For those who were able to map the iscsi lun on OF but was not successful connecting the luns after the scan in esxi. When creating the connection to the OF server, make sure you use the same “Target IQN” as state on the OF server. This should solve your problem.
[Reply]
One other comment: when you enter the ip of the iscs initiator host(s), make sure you use subnet 255.255.255.255.
Good luck to all.
[Reply]
I had stated that in an earlier post because it ties the single ip if you have your subnet mask as 255.255.255.0 you have a range of possible ip’s when it is set to 255.255.255.255 there can only be one ip assigned.
[Reply]
Hi all,
Thanks for your comments and tips for others – they are most appreciated.
Keep them coming.
Si
[Reply]
Excellent article ! I’ve had ESXi and OpenFiler running for months but try as I might I could not get the ESXi host to find the iSCSI target – it seems I’d set the volume up incorrectly, but thanks to your excellent instructions it’s now working – thanks very much for sharing.
Mick
[Reply]
Kiwi Si Reply:
March 19th, 2009 at 6:56 pm
Hi Mick,
Thanks for the feed back
I’m glad it is now all up and running for you.
All the best,
Si
[Reply]
Simon, this is splendid, thank you very much. Do you have a preference between the ML110 or ML115? I can’t really see a difference. Also, if you were to buy some storage for a home lab – any frugal recommendations? Is the Shuttle http://eu.shuttle.com/ your preferred choice?
Kind thanks
James
[Reply]
Kiwi Si Reply:
March 19th, 2009 at 6:28 pm
Hi James,
I have no real preference between the two of them TBH. They are both really good servers and ideal for the ole home/work lab. I guess the ML115 G5 (AMD) has the advantage over the ML110 G5 (Intel) at the moment as it offers a quad core CPU – as opposed to the ML110’s dual core.
I do like the Shuttle XPC’s because of their small size. I only used one for my filer as I had one lying around that used be my main PC a few years ago.
OpenFiler is pretty flexible when it comes to the hardware it’ll run on and doesn’t require a particularly large amount of CPU or memory resource. A decent 1Gb Ethernet adapter is highly desirable though.
If you have some old kit kicking around you could most likely get it running on that.
Failing that EBay has some cheap older shuttles for sale quite often. My shuttle is a Shuttle XPCSK21(G10) AMD model.
Let us know how you get on.
Simon
[Reply]
Great article!
Now I need to find a spare box to install Openfiler!
[Reply]
Great post. I have done a slighly different approach, using less Hadrware material (if you are on the budget…). I have used only 2 physical boxes.
Check this: http://www.vladan.fr/how-to-configure-openfiler-iscsi-storage-for-use-with-vmware-esx/
Vladan
[Reply]
[...] http://www.techhead.co.uk/how-to-configure-openfiler-v23-iscsi-storage-for-use-with-vmware-esx [...]
Excellent!
I was stuck for hours trying and retrying settings but it would not work.
Finally I found this tutorial and I saw right away that I missed the “Map LUN to Target”. As soon as I did that it worked.
Also, thanks for explaining the different types of volumes and that I could disable all other services besides iSCSI Target, that cleared stuff up for me.
Great job.
[Reply]
Awesome. This is the second post on this website that I have found EXTREMELY useful. Many thanks for that. I followed these instructions and everything just worked.
Now, how about a post on how to add OpenFiler to a Windows 2008 Active Directory
Thanks Again
Richard
[Reply]
Kiwi Si Reply:
April 30th, 2009 at 7:11 am
Hi Richard,
Thanks for stopping by TechHead. Glad you found some of the posts useful.
Hmm, a post on adding OpenFiler to W2K8 AD could be good. I’ll add it to my list of articles to write.
All the best,
Si
[Reply]
Thanks for a great article, just starting to work up a Windows 2008 cluster. Really appreciate the guidance on OpenFiler, iSCSI and vmware.
[Reply]
I also had problems finding the luns (keep getting 0 targets) untill I edited the /etc/initiators.deny file on the openfiler.
Just open with VI, retyped the last word ALL and saved and then it worked.
THX everyone for the help.
[Reply]
Worked like a charm
Thanx a lot m8
[Reply]
I need to have a single volume for my VM greater than 2TB. What is the largest single volume you can make with an iSCSI solution? I think ESXi can only see a LUN up to 2TB, but I thought you could make a bigger datastore than that. Has anyone had experience over 2TB?
[Reply]
lukasware Reply:
May 9th, 2009 at 12:12 am
OF will export an iSCSI LUN greater than 2TB
(i’ve done 5) the OS have to support GPT partitioning (usually 64 bit OSes do including XP64 and Vsita 64. VMware ESX does not and it’s 32bit until ESX 4.0 (end of may?). But you can span mutiple 2TB VMFS together I think…
[Reply]
[...] 2. http://www.techhead.co.uk/how-to-configure-openfiler-v23-iscsi-storage-for-use-with-vmware-esx [...]
[...] http://www.techhead.co.uk/how-to-configure-openfiler-v23-iscsi-storage-for-use-with-vmware-esx [...]
Anybody tried Openfiler 2.3 with vSphere 4.0 yet?
[Reply]
Heiko Verlande Reply:
June 8th, 2009 at 11:17 am
@whtech Yes, i installed ESX 4.0 with Openfiler 2.3. Read about it [url=http://blog.verlande.nl/2009/06/free-shared-nassan-storage-using-openfiler-23-nfs-on-vmware-esx-40/]here[/url].
[Reply]
Kiwi Si Reply:
June 9th, 2009 at 7:51 pm
Hi whtech, I sure have and it works just fine. In fact I found connecting to the Open Filer iSCSI target from ESX 4.0 to be quicker.
[Reply]
OF works great with ESXi 4.0, make sure you get the 2.3.1 “respin” which has 2.6.26 kernel as this installs better on modern hardware. I skip all that network access stuff and just create the iSCSI lun (as above, but with write-back and fileio) and then map it (as above). Saves time because the ACL was jsut another barrier to getting vmotion and vstorage motion working. with ESX/i 4.0 3x I/O performance improvement and 10x iSCSI performance improvement (and finally support for jumbo frames) makes sense to pretend ESX3.5 never existed.
[Reply]
[...] ESX hosts (physical and virtual) are connected via iSCSI to a machine running OpenFiler and the storage networking works ok, however when I configure the vMotion & FT private networks [...]
Ok, so I easily got iSCSI up and running with these great instructions. Well except for one problem – I’d like to have two LUNs. One for a 2TB volume and the other for a 1TB volumes.
I’ve tried just mapping the two LUNs to the one target without luck and then tried creating two targets and mapping one to each. No matter what I do I can’t get the second LUN to show up in the iSCSI Software Adapter.
I’m using ESX 3.5.4 and Virtual Center 2.5.4. I’ve tried editing the initiators.deny file as suggested without luck.
Does anyone have any ideas or is this something that isn’t supported by Openfiler?
Thanks.
[Reply]
Nyle Reply:
July 2nd, 2009 at 1:33 pm
Never mind, I got it working by removing all the configuration after a full upgrade and reboot and doing it all again.
Then I found out about VMWare LUN size limit. My 3TB LUN only showed as a much smaller storage location. So, I undid the larger 3TB volume and broke it into two separate 1.5TB volumes. Then mapped those two to the target.
I now have three LUNs – 1TB, 1.5TB and 1.5TB.
Once all my ESX servers could see it I then added a VMFS 2 storage locations. The second one I added the second 1.5TB LUN to as an extent.
Now I’m fully in business. Thanks for these great instructions.
[Reply]
Kiwi Si Reply:
August 18th, 2009 at 7:10 am
Hi Nyle,
Glad it worked out ok for you.
Thanks for letting us know of your experience as this will be most useful to others.
Thanks,
Simon
[Reply]
Hi,
great article. Got it working for the most part. However, we only have 1 nic and we tried to associate that as well to the virtual network. Once we did and hit ‘ok’, we lost connectivity (VM Client, ssh). The machine seems to be up and running as we have a sql server on that box as well and it seems to be up.
Can you offer any suggestions.
Thanks
Patrick
[Reply]
just as an after thought to my initial post – we don’t want to reboot the box in case it doesnt come back up. We do have console access at the box, so we are hoping there is some way to delete the changes we made.
Thanks
[Reply]
Kiwi Si Reply:
August 18th, 2009 at 6:54 am
Hi Patrick,
I’ve only just read your message. How are you getting on with the issue you mentioned?
I wouldn’t recommend using a single NIC on your ESX host for both your virtual network and iSCSI traffic. If you don’t have any other choice then you would should consider using VLAN tagging to split the two traffic types up. Are you also using this single NIC for your management traffic to the host also (ie: VM Client, SSH, etc)?
Let me know how you’re getting on.
Cheers,
Simon
[Reply]
pfitzpatrick Reply:
August 18th, 2009 at 4:35 pm
Thanks for the reply Simon.
We made significant progress yesterday/today…in that we managed to get the box back up and on the network with all the services available.
We made use of command line config files to delete any/all references to the new switch/virtual net(esxcfg-nics/esxcfg-vmknic/esxcfg-vswitch, for everybody’s info. These files are located under /sbin).
Once we deleted the references, we rebooted the box, but we were stumped/gutted for a minute when nothing was working. Turned out that when we went into the network/nic card administration, the nic card was not enable. Checked that and we were suckin diesel again.
Thnanks again Simon.
[Reply]
Here’s my issue. I have 5 nics in my OF Server. 1 Built in 10/100 I have setup with 192.168.0.x and it is connected to my main network for console use only. I have 4 other nics with 192.168.50.x it’s on a separate network and I can not get ESX to see the iSCSI Lun. If I put the 4 nics to the 192.168.0.x network it works. I have a vKernal setup with 192.168.50.x 255.255.255.0 and it is using a dedicated NIC on the Esx server.
[Reply]
Kiwi Si Reply:
August 18th, 2009 at 7:06 am
Hi Jason,
I would recommend looking at splitting your iSCSI (OpenFiler) traffic off onto a seperate subnet and if using the same physical switch as the other traffic (eg: VM traffic) then look at using VLAN tagging. Although this is not necessary it will help keep things seperate.
I’m not saying that this is cause of the issue you mention as if (correct me if I’ve misread this) you put your iSCSI port on the 192.168.0.x network you can connect your ESX host to your iSCSI based LUN presented from your OpenFiler server?
If this is the case then your OpenFiler server has it’s network port (assuming it just has one) configured to the wrong subnet (ie: 192.168.0.x and not 192.168.50.x). How I think it should look:
Build in NIC –> ESX Management – 192.168.0.x
3 x NICs –> 192.168.50.x – VM Traffic (or perhaps 2 x NICs for VM Traffic and then 1 x NIC for vMotion)
1 x NIC –> 192.168.60.x (new subnet) – iSCSI traffic. Ensure that the OpenFiler iSCSI port is also configured to use an IP on this new subnet. You could always use one of the 4 NIC ports you’ve got as a failover NIC for iSCSI traffic – though you’d really want seperate physical switches to get the most benefit from this resilient configuration.
Hope this makes sense and that I read your message ok.
Cheers,
Simon
[Reply]
jasontech Reply:
August 18th, 2009 at 4:42 pm
Here’s my setup
[Reply]
Simon,
Excellant documentation. I managed to figure out how to do it myself before I found this article but like the guy above, I can find the iSCSI LUNs when I use my main Vswitch on the two ESX3i servers (172.25.120 mask 255.255.0.0) but I have a separate Dual Gigabit NIC in both ESX3i servers and also in the OF Appliance. I setup one interface in the OF as 192.169.0.69 and the other as 192.168.1.69 and one ESX3i server has a NIC setup as 192.168.0.6 and the other ESX3i as 192.168.1.4 with direct cable from Openfile from each ESX3i server to the Openfiler on the relevant NIC. I can Ping each ESX3i server from the Openfiler console but when I try and discover the LUN after changing the Target in each ESX3i box to the relevant 192.168.0.69 or 192.168.1.69, it fails to find the LUN but if I change it back to 172.25.200.69 it finds the LUN straight away. Any Ideas.
[Reply]
Hey,
I really want to thank you for such a good step by step article to setup iscsi on openfiler. I really appreciate your efforts and time spent on this article.
regards
Leon
[Reply]
Hi Simon,
First off, thanks for you posts. I’ve learned much because of what you write. thank you thank you thank you.
Now, I was reading through your openfiler setup with VMware, and i was wondering you already run into this issue: I have 4 ESX host. I can only connect 3 host to openfiler using iscsi. Everytime I connect the 4th one, one of the connected 3 will lose its connection to isci, and if I look into the vmkernel logs, it would display something like:
Sep 17 17:27:17 esx22 vmkernel: 0:07:24:41.099 cpu0:4194)WARNING: NMP: nmp_DeviceAttemptFailover: Retry world failover device “t10.F405E46494C454003363162625F6D266072393D23333C677″ – issuing command 0×410004034e40
I thought the solution was to change the MAXConnection setting in openfiler, but the default setting is “1″ and I can successfully connect 3…so I’m a bit lost.
Hope you can shed some light into this.
[Reply]
excellent post, thanks for taking the time
[Reply]
Great post.
:)
Thanks, had this set up in 2 hours after reinstall
[Reply]
GREAT GUIDE !!!!
That is what I have been looking for a while, found parts and bits but not a very complete guide that focus on the confusing part of the OpenFiler process (Volumes).
I hope the OpenFiler folks hire you to create their next guide.
I have a question regarding the subnetmask.
I have a lab network and I use 192.168.1.x ip address range with 255.255.255.0 subnetmask all my VM and ESX hosts communicate with no problem between eachother but I read in your forum that people had problem making OpenFiler see the ESX server when they use this subnetmask and had to change to 255.255.255.255 to work. Will I need to change mine subnetmask so my ESX server can see openfiler ?
Thanks for your asssitance.
Neo
[Reply]
Hello
One more question.
I did not find yet in any guide that would give me the answer to my question below:
Scenario:
I have Windows 7 running Vmware workstation 6.5 I created inside of the Vmworkstation 2 ESX 3.5 (VMs) on it. I want to install OpenFiler and make the 2 ESX servers (vms) see the shared Storage so I can test VMotion, DRS, HA features.
I downloaded the iso image for OpenFiler 2.3 x86 and I want to install OpenFiler through my Vmware Workstation.
Since there is no instructions in how to do this I believe you would need first to create a NEW VM option on my Vmware Workstation 6.5.
There I selected linux 4 64 bit and so forth. And selected install OS through the DVD (my openfiler iso I burned on a DVD).
My QUESTION is during the VM creation when it gets the time to select disk space size it usually shows 8 gigs (default). But since I’m creating Openfiler and I want to have at least 50 gigs of disk space to be shared and used by the 2 ESX hosts.
Is it here on this step where I would change from 8 gig to 50 gigs and then after OpenFIler is installed that I would go inside and configured the 50 Gig diskspace into shared storage ?
The above step I did not find anywhere yet. So if you guys could assist I would really appreciate it.
I read one guide that said that he had a OpenFiler.VMK and he used it and did not explained the settings that he selected or were selected when he used the OpenFiler.VML
I tried looking to download this VMK file but it does not seem to be available anymore.
The steps that I did above creating a VM, and Disk space, etc are these the recommended ones that you guys also had to do it or do you recommend another approach to install OpenFiler and have the 50 gigs disk space available to be shared between ESX hosts (vms). ?
Thanks for any assistance on this matter.
Neokanobi
[Reply]
[...] L’installazione del prodotto, inclusa la configurazione delle ACL di rete, la creazione dei volumi e l’esposizione delle LUN ha richiesto circa un’ora e mezza. Per una guida passo-a-passo vi consiglio l’ottimo articolo che ho trovato a questo link. [...]
Social comments and analytics for this post…
This post was mentioned on Twitter by delicious50: How to configure OpenFiler v2.3 iSCSI Storage for use with VMware ESX. | TechHead.co.uk http://bit.ly/zZl6E VMware Storage…
[...] This post was Twitted by delicious50 [...]
Hi,
2 questions:
1. How many disks does the shuttle PC have?
2. Can you use Openfiler with just 1 physical Disk (I realise you could not use RAID) just to test it as a SAN storage for 2 X ESX(i) Hosts and test HA etc?
Thanks in advance.
[Reply]
Kiwi Si Reply:
December 22nd, 2009 at 7:59 am
Hi there,
My shuttle has two disks – though I have recently moved my OpenFiler to a ML110 G4 as I can fit up to 4 drives in it.
You certainly can use OpenFiler with one disk. The ESX/ESXi hosts you connect to the shared disk that the OpenFiler presents don’t mind whether this disk is in a RAID configuration or not. All they (the ESX/ESXi hosts) care about is that they have some shared disk and from there you can test HA, DRS, FT (CPU permitting). Of course, as you no doubt know, by not having RAID you are not protected from physical disk failure and coule lose everything if this was to happen.
Hope this helps.
Cheers,
Simon
[Reply]
rp2000 Reply:
December 22nd, 2009 at 11:35 am
Cheers for the quick reply.
At present I have bought 2 ML115s to use as the ESX(i) Cluster. I am not sure If I can get access to a 3rd machine that has more than 1 disk drive (hence my question). Ideally I would like a 3rd OpenFiler box with 2 disks to use RAID, but may have to settle for a single disk, proof of concept, machine to begin with (maybe just an old desktop, if I can get hold of one).
I may have a HP XW4300 that I can use as the OpenFiler at a later date and I think this can hold 4 disks etc, although (physically) it is bigger than a ML115!!
[Reply]
Kiwi Si Reply:
December 22nd, 2009 at 9:44 pm
Hi,
Another option would be to run a storage virtual appliance (VA) on both of your ESXi servers which syncs the data between both storage nodes. OpenFiler can do this and I will be demo’ing three other storage VA’s that can also perform this role – check out my post here for more
details: http://www.techhead.co.uk/building-your-own-vmware-vsphere-lab-a-step-by-step-guide-part-1-lab-overview
Cheers,
Simon
I’m from Brazil. Thank you my friend, helped me a lot in my lab.
[Reply]
We are from germany.
Opensource is great. A good howto is the greatest.
Thanks a lot.
[Reply]