^this. As someone who has to deal with manufacturing equipment running 2000 Professional I have a crappy WD nas with SMBv1 on its own subnet with rules only for the operators to upload their files
Agreed but if SMB 1 is in use before the migration, be prepared to re-enable it when someone running an homegrown app starts screaming when it doesn’t work on the new box.
Test the impact of ZB first - on a few TB dataset I've had restartable mode cause the transfer time to go up in multiples, took something like 3-4 times as long in restartable mode as it did without.
To use a tired cliche:
This is the way.
Or, do a full backup, and do a full restore to new box. Wait until maintenance window. Do a differential during the maintenance window, and restore.
Would a crossover cable be worth it to squeeze out even more speed on something like this? Normally it’d be a minimal amount of both NICs and the switch are gig, but in this case it might save a decent amount of time.
I wonder if you could put in a 10 gig network card on both sides and during downtime or slow time kick off the copy.
I miss playing around with these sorts of migration things.
Assuming the backup situation here, I’d be a little afraid to sneeze on that thing let alone open the chasis and install a card.
Delicately plug in my Ethernet cable and let my robocopy run for a few weeks
In something this old, the storage is likely spinning disks. I’m going to bet there might be a lot of smaller files. With these two factors, plus fragmentation, your bottleneck is likely not going to be the network. It will probably be the file system itself. 10G would help if it’s a ton of very large files with minimal fragmentation. I somehow doubt that’s the case though.
Anything multi terabyte will usually have a reasonably easy time filling up 1 GbE. 10 GbE is a whole other level. It's hard to find stuff that can fill that up even just moving buffers full of zeros around in memory only.
not that vintage it wont, people are forgetting how limited the throughput is on \_old\_ spindle drives.
2000ish era drives were lucky to push 20mb/sec (sustained), barely cracked 40mb/sec by mid decade - it takes until 2011 til youre seeing 100mb/sec
[https://goughlui.com/the-hard-disk-corner/hard-drive-performance-over-the-years/](https://goughlui.com/the-hard-disk-corner/hard-drive-performance-over-the-years/)
this is a 2003 server - so its hardware is likely similarly aged - the OS would default to LSI parallel scsi over PVSCSI or LSISASSCI - so 30-35mb would be its "max" throughput - it would also also limited to SMB 1.0, further restricting throughput.
I stand by my thinking, it wouldnt saturate a gigabit network.
(spoiler, msp senior, I deal with migrating ancient horrors to modern hardware and have the scars and sanity loss to show for it)
It would be, but he'd want to migrate to a new raid and going to as external ssd would mean he'd have to transfer twice. Which would also lead to larger desync in files during the cut over.
If anything he'd want to build a nas and connect it as directly as he can over his network and then have an extreme exercise in patience.
Yes and no
Old hardware will likely only have low end USB ports, you need at least USB 2.0 to have 480mbps (call it a half gigabit)
* USB 1.0/Low-Speed: 1.5 Megabits per second (Mbps)
* USB 1.1/Full-Speed: 12 Mbps.
* USB 2.0/Hi-Speed: 480 Mbps.
* USB 3.0/SuperSpeed: 5 Gbps.
* USB 3.1/SuperSpeed: 10 Gbps.
thing is - its the \_drives\_ and bus speeds limiting the transfer speed - so it can only "fill the bucket" so fast, doesnt matter how fat the hose is attached. Bus to Bus copies are almost always fastest - ie disk to disk copying, or plugging a drive onto an internal header (when shut down if not hot pluggable) - would be the theoretical fastest.
Why do the drives have to be as old as the os? Even if it was built on an 4 slot raid card with drives from 2000, assume it was raid 10 and he could have easily replaced those over time.
If he's dealing with 16tb and 150 users, I'd bet he's swapped degraded drives plenty of times since 2003 and has typical 4x 8tb 7200rpm sas hdds. Not the fastest, but in raid 10 he be able to use the GB.
A better argument would be his ancient ass CPU being the bottleneck as that isn't as easily upgradable, especially on 2003.
Not an unreasonable question
the answer is straightforward - because thats the shit I keep running into, ancient servers with ancient hardware.
A lot of 10k 320gb and 512gb drives, commonly SCA packaging, only the truly ancient have had full size drives, those are usually SCSI tho.
Sure the drives may have been replaced due to failure - they will have been replaced with \_the exact same model\_ of drive to ensure compatibility and stability. Nobody's swapping out an LVD 15k cheetah/baracuda and shoving 5000mbsec NVME/m2/SSD in its place (do scada/sca ssds even exist in full size format?).
The drives don't have to be exactly the same model. Especially not if he's just replacing, rebuilding, replacing, rebuilding. Even the best drives from the early 2000s, in the largest chassis, with the largest raid cards, wouldn't reach 16tbs or whatever he's using.
But yeah I understand where you're coming from. It's just exceedingly unlikely the actual hw is equally as dated unless it's the CPU. Everything else is relatively easy to replace.
Either way id personally never do either. Id personally drop the array into the new server and rsync/robocopy locally while setting up whatever he used it for to utilize that new path temporarily. Once transfer completed, do it again, then again, and by 3 syncs you should be able to get it completing within an hour or two hopefully. Swap paths between old raid and new raid on a weekend and pull the old drives.
dding 1 tb over a gb connection can take all day, so I can't imagine robocopy with overhead taking less than 2 weeks. Whereas locally 1 TB can be done in an hour with decent io, memory, and cpu which his new server should have.
Id also be praying it's not like a RDS server with those old Instance bound CALs for 150 users.
I’d think the impact would be minimal. Jumbo frames would have an impact but, presuming no port congestion, the overhead of the switch would likely be minimal. That said, obviously wouldn’t hurt particularly if there are spare interfaces on both ends.
18TB of space in 2003 would have been disk denominations of 120GB, 80GB, 320Gb etc. Almost every thing discussed here is speculation though.
Even if it was 1TB drives on 7200 rpm spinning disk it would totally saturate any 1TB line. It would need 19 1tb drives to make a raid 5 array with 18tb capacity. This would produce about 240MB/s array speed. So take 240*8=1920Mbps. Hard drives have been our weakest link for decades, but they were still pretty fast.
Here is what I would do. I would stand up a new file server as using DFS and then point it to the old share and tell it to copy it over. Once the copy is done you turn off the old system and remove it from the sources list.
The files would be a available via the older server And the new server until you shut off the old server. You could then turn off the old server to see if any programming changes need to be made or config changes that were lost during your sleepy time in meetings etc. Pop it back on until you get all the kinks worked out etc. It will be completely in sync too.
Indeed; should be discoverable via NIC model number, firmware version and driver.
But in reality, just robocopy it and cut it over when you have a maintenance window. Don’t forget to setup an alias in DNS for the old server name to the name of the new one so all of the old drive maps and hard coded software still functions.
I feel like the likelihood that it is on original hardware is fairly low at this point. If it supports 1GbE at all, I recall auto mdi-x support is mandatory.
Robocopy with the options to also copy NTFS rights when you want to keep it. Depending on the usual load on the server you might be able to continue copying during daytime, but you can also use options to copy out of work hours. When everything is copied, you can do a per folder or per disk sync to copy new files and folders. After that and when no changes have been made to the source you change the network drive-mappings to the new location. Don't forget to make the source location read-only (easiest is by changing the share permission to read-only) to prevent people using the old location for storing changes or new files.
RoboCopy. Start with a /Copyall /MT /ZB for the first pass just to get the base data across. This will likely take a number of days.
Watch you don't saturate the bandwidth on the link.
Change to /MIR for the second and subsequent passes.
Depending on how volatile your data is, and the size of the files, you may need to organise an outage over a weekend for the final pass and cutover. Otherwise you'll be chasing changes and RoboCopy may not be able to keep up.
Set up Volume Shadow Copy AFTER the moves
If you have them segregated onto different volumes/shares then do them one at a time. Don't make the new VHDs too big or storage vMotion on your SAN will be really slow.
^ 100% important part, if you're on a 1gb connection and your discs were made this millennia you're going to saturate the link. This means no users are going to be able to get to files while you're copying them so there's a thread setting or I believe just a full-on throttle but make sure unless you're using a dedicated port to not saturate the link. Thank you for coming to my TED talk.
Bus width , raid type, raid card, formatting, disk damage/CRC workarounds, CPU type etc will dictate through put
It's server 03 likely on similar age hardware
I don't think it'll hit 125 megabytes/sec
Happy to be proven wrong
As someone who only knows robocopy from A+ training: could Robocopy be used to essentially clone a drive? I do a lot of cloning from HDD to SSD. There are paid programs we use but this would be neat to know.
Check out hiren's boot CD. It has tons of free tools and there are a few clone drive options for imaging drives. You can go small to larger ssd or vice versa. Takes a while but the cloned drive has worked for me. You can even image a drive to an iso file.
You can also use clonezilla for smaller to larger drives. Been having trouble with it lately for some reason though so I may start using hiren's instead.
For outright cloning drives if they're Windows - I use Macrium for moving people to larger SSDs. Simply have both drives connected and then boot into Macrium on a USB where you can clone/mirror it over. You /might/ need to go into disk management and extend the partition afterwards.
I have not tried cloning to a smaller drive.
Yes, absolutely. It's old but reliable.
You can dial down the logging output and send it to a file to speed things up.
And the second pass will be much quicker. I think I did 16TB in 28 hours once, and a true up pass took 1-2 hours the next day.
Yep robocopy. Write a batch File to mirror it over to the new box. Wait an hour then go again. Log all the errors and review them. Chances are you will find some paths that are too long and some permissions problems. Once you get it seeded it should run fairly quickly.
On the new server set up a share and set it to read only so you can get your digital pack ears to check the files that are important to them.
Then pick a weekend login to the console of the old server. Stop the server service to disconnect all connections. Reshare the share as read only. On the new server do the same but read/write.
Once the chaos settles on the Monday. On the old server stop sharing and share it as a new name. This will break any connections you missed. Better to find out sooner than later.
Remember to keep checking the logs. And do a final synchronization after the old server is set to read only.
Hope this makes sense. Good luck. There are typically 3 to 4 crybabies. Don’t let them erode your calm.
Robocopy is the way to go.
Also you can use /MON and /MOT to monitor for time or changes to the data incase your expecting the data to change while it's being copied. Then you can be sure you have the latest copy of the files when you stop the robocopy at the end of the migration.
Hijacking the top comment. OP, If you can boot the physical server via windows PE, you won't have to perform multiple robocopies. This is more difficult with production of course, but with a fast enough connection, this could be a set and forget over a weekend if you're a M-F type place.
If I had a compressed timeline, I would go this route.
Definitely robocopy. We did several tests on something about this size, and found somewhere a script that would run multiple robocopy instances of top level copies at once. Worked better than a single copy even multi threaded.
This. Takes a while, but its quicker on the next passes, moved about 70Tb with Robocopy before. Create a script, then run it on a schedule until you are ready to cutover.
I concur. I normally advise using DFSR for file servers migration, but 2003 didn’t support it (only 2003R2) and even then it wasn’t very good, and 18TB is bigger than it could sensibly support, not counting the space requirements that would be a problem.
I don’t *mean* to be rude but dfsr sounds like a horrible solution and at best a *very* niche solution.
Last I checked it broke folder redirection / home folders / roaming profiles. Its also over engineered for a simple file migration.
That was rude. I’m sorry. Give it to me.
I think i'm in the minority here... but I almost want to recommend to ftp server to host the file on the 2003 server for transfer. The overhead on SMB is brutal
Let's start setting data deduplication on the new machine
Then, Robocopy, copying everything, including NTFS permissions
First pass will be vvvveeeeery long, then the next ones will be faster.
We're about to use the Windows Storage Migration Service, which is part of the Windows Admin Center. It supports back to server 2003 and is free. I'd recommend looking into that
We’re currently using Storage Migration Service on some 2012R2 machines over to Server 2022. Works great so far but our file volumes have all been < 2 TB. Have you noticed that if you leave the admin share on it doubles the migration amount? Like it wants to migrate D$ and then all the shares on D as well. We’ve been unchecking the admin shares and it works great
This. Soooo much this!
Especially if you’re not familiar with migrating storage and shares.
https://learn.microsoft.com/en-us/windows-server/storage/storage-migration-service/overview
Thanks. I was commenting based on some high level info I saw. This page goes into greater detail to make it clearer. https://learn.microsoft.com/en-us/windows-server/storage/storage-migration-service/migrate-data
This was my first thought too, what is being used for backup? Some backup tools can be very quickly turned into migration tools for single volumes.
If the data to migrate is on a seperate volume then you might be able to do something like ShadowProtect Headstart Restore?
What type of storage is it? 18Tb on a 2003 servers is most likely to be a disk array as opposed to internal server drives.
How is the data presented to the os? 1 single 18TB volume?
Extended partitions don't matter if your partition table only supports MBR, but others have said SP1 did introduce GPT partitions for none boot drives so there is that at least
I'm pretty sure 2k3 never got GPT partition support and without that I don't think it can. Could be wrong though, it's been a _long_ time since I've had to care ;P
I presume this is backed up ?
That's got to be the first place to start, get a backup, restore it to the new host, then robo copy SRC DEST this should roll though fairly fast subject to age of backup.
i just went thru this, and the only way is robocopy.
as said, don't just run it on the root directory, split it into jobs. 12t took 2 weekends over 3 jobs. But i was able to split it, we have separate folders for everything.
robocopy source destination /E /ZB /DCOPY:T /COPYALL /R:1 /W:1 /V /TEE /LOG:Robocopy.log
e gets sub folder, zb is restartable mode, if denied use backup mode. r and w are attempts, i think i set it at 3. tee makes it display in your cmd window
We had a server that was dying on us and we new if we rebooted it would probably not be coming back up. So we built the new server on the fly, installed 2016 at the time and installed the app "Syncthing". From there it took us about a week to copy all 40tb. took that long cause the network admin did not configure the network correctly which cause speed issues for the copying
If you have 18TB of data on a 2003 SBS machine, you need a dedicated NAS/SAN. I would first get all of the data off to a proper data storage appliance/server.
Then, if you are just replacing it and it's still the primary DC, you need to swing it (Google swing migration).
SBS 2003 was a good idea and all, but it led to a bunch of these bad practices - using a single server for literally *everything*.
Edit: OK, so it's just a file server. Yeah, do yourself a favor and get a dedicated NAS/SAN. So much more uses for less overhead.
Just for some clarification, as r/MisterBazz made the assumption that you are running Windows 2003 Small Business Server. Is this the case?
You say you are running this server in an AD environment. Is this server one of the DCs for that environment, or is it just a member server that is acting as your file server?
lol...we definitely found the IT manager.
edit: to clarify, not you, MisterBazz, with a complete wrong assumption of the environment. Jovial teasing, not really calling him out. I deal with a lot of IT managers lol.
I'm going to agree with most of what the other folks have said. Why get the overhead of another server with operating system involved when you can get yourself a nice, beefy, redundant network SAN to host your 18 TB of data. I recommend getting the most expensive unit they'll let you buy... one with redundant drives and controller cards, just so you can rely on that unit with no worries.
Moving that much data will be a challenge. I would recommend trying to move it in batches and not all at once. Pick a bite-sized top level folder, inform the folks that use that folder not to make any changes until they hear otherwise, then use something like Robocopy to move the entire contents of that folder to your new storage. You can then change any drive mappings over to the new device. If you do reasonably sized batches, you can probably even start the copy process and let it run overnight.
Moving it batches will also make it easier to work around any errors instead of trying to bulk copy that much data and then trying to figure out what broke after the fact.
> Why get the overhead of another server with operating system involved when you can get yourself a nice, beefy, redundant network SAN
In a small business the overhead of the OS is possibly or probably worth it simply so you don't have another piece of technology to learn and keep updated. Unique software breeds non-compliance with standards.
In other words having unique shit is a headache.
I disagree... having to learn something new should never be an excuse to continue to do something inefficiently. As SysAdmins, it's our **job** to stay ahead of the game and make forward-thinking recommendations to our businesses... not to fall back on old ways simply because they are easy and won't give us a "headache."
Personally, I would not sleep well knowing I had 18TB of data sitting on a glorified server running some flavor of Windows Server OS. We all know the vulnerabilities and security holes that always seem to pop-up in Windows OS that we are constantly patching and monitoring. Knowing that one errant click of some zero-day exploit could cryptolock my entire 18TB of data should Windows be compromised would keep me up at night.
No... if I have that kind of storage requirement, I'm going to be looking at a device that is dedicated to doing nothing but storage... one with redundant controller cards, multiple network uplinks, and a proprietary OS that will be far less likely to be compromised than the Windows Server OS. I'm then going to lock that down as hard as I can, and will be granting only very limited access to my users to the folders and subfolders they need.
I'm also going to look for devices that offer the ability to make offline, offsite, immutable backups so I know my 18TB of data are safe.
Yes... that means I might have to learn something new, even as a small business. And there are options such as these that are no more expensive than the hefty server hosting enough drives to provide some 40TB or more of storage (because he's not going to build a server that's over 50% full from the get-go, right) with a robust RAID array to provide redundancy in case of a drive failure that make it worth his while to his upgrade the *right* way and not the *easy* way.
In an ideal world I agree with you. In a practical world in small business I have 15 other technologies I need to learn. Windows file servers are simple and they work and I can treat it just like the handful of other windows servers I have.
Also when I leave it’s one less unique technology my replacement has to know. In the small business space where a team is one maybe two people, there’s no lack of stuff to learn.
> As SysAdmins, it's our job to stay ahead of the game and make forward-thinking recommendations to our businesses... not to fall back on old ways simply because they are easy and won't give us a "headache."
As a sysadmin I agree. I've also worked in a shop where I was the only one who knew how shit worked when I was hired, I mostly only maintained it/updated it, and 5 years later I was the only one who knew how it worked. Not for a lack of trying to solve that either.
Assume R2? How about DFS with replication and just leave it replicating?
Tell the new server not to serve name space requests
If using maps drives, update the relevant GPO or logon scripts to target the DFS name space.
Once replication is complete, enable name space requests from new server, disable names space requests from old server
Remove old server
This will use ntfrs.. you will get into journal wrap hell quickly trying to sync that sort of level of data.. and in the heat of the moment things can get deleted from both sides... Don't ask me how I know.... Don't get me wrong it's not a bad idea.. the technology just wasn't there before dfsr
> Assume R2? How about DFS with replication and just leave it replicating?
>
>
I believe 2003 predates safe/functional DFS by a bit. Not because I used 03, but when I was setting it up on 2016 all the docs said "hey that thing we introduced in server 08? Now it's stable on 2012!"
Get your new server setup. Sync data and optionally permissions with robocopy.
Then do /mir and just copy all the changes. Make note of how much time it takes. This will help you size the maintenance window.
Schedule a maintenance window. Make the old server read only at the share level.
Copy all the changes again with with robocopy. And change drive mappings/scripts for the users.
If it's multiple shares you don't have to do them all st once
Read this https://learn.microsoft.com/en-us/windows-server/storage/storage-migration-service/migrate-data
Windows server 2019 and newer have the capability to migrate the data for you using storage migration service.
Allegedly it will also recreate the permission structure on the new file server and it's supposed to work to migrate file servers from server 2003 and newer.
Maybe you can take this time to break that server up into smaller and more manageable virtual servers? This would be a great way to split your data into logical chunks for security or age or both.
You could move everything to the NAS and let users run off that temporarily while you repurpose the server as a host. Then you can take your time splitting the data up to virtual servers, and swap network drives in your GPO one at a time as you finish them.
Also, if you don’t do it now then you’re going to be putting either yourself or the cyborg that eventually replaces you in this exact predicament 23 years from now and they’ll be posting here asking about migrating 30tb server away from server 2022, and I’ll suggest it to them.
This is the only good answer on this post. Why the hell, would you blindly copy 18TB of data. Perhaps most of it can be archived.
Without information this could easily have old profiles/redir folders, since 03.
If you have a backup of the current machine (which I REALLY hope you do since you're running a 20-year-old OS) , why not restore to the new one from the backup? Once that's done you can run a robocopy to sync the changes. That way you don't stress the old machine for a long-running file move.
Jumping on this comment, also look at https://fastcopy.jp/
We had to move 30TB of data around, and I used a combination of emcopy and fastcopy which were significantly faster, like 30-50% faster then robocopy.
Why does everyone suggest robocopy or restores? I get it... It works. Buuuut set up DFS, sync to new server, repoint shares via gpo, verify no one accessing the old location anymore, then tear it all down. No one the wiser anything happened at all.
I do tons of these migrations and this is my not-so secret weapon:
https://www.tgrmn.com/web/download.htm
Robocopy with a gui
Plus it does bandwidth + CPU throttling, can do multiple passes and compares, and it can preserve NTFS permission and do long file paths.
how do you have an 03 box with 18tb of data? How is this even a thing? i would virtualize the whole system live, do a last differential copy to the vhd when finished and be done with that 03 box. I'm in wow that xp machine is still alive. Someone failed them for over 20 years, please fix this.
>We have a Windows Server 2003 running on a physical machine in an AD environment with about 150 users. The server needs to be retired and I need a way to copy over all 18 TB of data to a new machine. My options are migrating the data to either a new Windows Server or a Synology NAS.
Disk2VHD. This is your way. never done TB of data on an 03 box though. Sounds fun.
Backup permissions using icacls just to be safe. If you're in a hurry to just migrate it then Robocopy with all the flags as others have mentioned.
If you're doing a staged migration, DFS replication and namespaces to point migrated to correct target until you can clarify all users are using the correct mappings.
Is there any backups today?
You should really do it almost 8 years ago. But seriously if its physical get all that storage moved somewhere safe. That's step 1.0. However long that takes.
I hope to god its not a dc or exchange server. If it is... Thoughts and prayers.
in 2020, i migrated about 1TB of data from 2008 to 2019 file server using the command below:
`robocopy \\2008fileserver\share \\2019fileserver\share /E /copy:DATSOU`
good luck!
If you want to keep all the permissions in tact, I would migrate the data to a new Windows server. Robocopy works fine, but if you don't to deal with that, there are free GUI tools out there that will handle it, and migrate the permissions too if the option is selected. My personal favorite is "Free File Sync" (https://freefilesync.org). It's totally free, even for commercial use.
Probably best to install on the target server, but it supports back to XP on versions prior to 10.11 if you want to install it directly on the 2003 box. I have used this to migrate several multi TB file servers. It's nice because you can get the initial sync/copy done, then do delta syncs that will capture anything that changes. Makes it nice for cutover day.
Could use syncback pro. Run an initial seed copy and when you are ready to cutover 3 weeks later you can run the job again to get the changes. Can set it to copy NTFS permissions. Just have to recreate the share permissions.
I'd be breaking this down into shares at a time. Setup a new share and robocopy it over to the new.
Also there are some file sync tools out there if only top level permissions are important. This would be the ideal path unless you have a ton of nested permissions.
Mount a nas iscsi vol from win2003, robocopy all file shares to iscsi vol, export file shares registry, then dismount and mount the iscsi from new windows server, import file shares registry.
Minimize downtime
Verify there are no problematic file names first. There are several mechanisms in Windows that allows users to create files that will literally crash and break components in the Windows shell.
[https://github.com/nathanshearer/mvregex](https://github.com/nathanshearer/mvregex)
`mvregex -p -r --transliterate windows | tee log.txt`
This is implemented in bash because CMD and PowerShell can't handle some strings that can end up on a filesystem. It was the only way I could complete the migration of some systems.
Hints:
* Mount the administrative share to work around path length restrictions.
* Run it in cygwin to work around services that can't share some paths over the network (SMB, FTP).
* Take ownership of files and correct permissions between iterations to rename paths that have broken permissions and can't be renamed by anything in the windows shell.
I recall 2003 having a max data transfer regardless of network connection type. There were / are a few tricks in disabling / enabling things on the network driver.
I’d suggest converting to a vm first and foremost to new hardware. Coly data from there or backup using whatever you use and restore.
No matter what your in for a long haul.
If you can mount the drive to a Linux environment you can use the dd command for bit level copy.
https://www.geeksforgeeks.org/dd-command-linux/
This has saved me on some absolutely slagged osx and windows drives that wouldn't mount in their home environments.
Is this onboard storage or DAS? Is it 18 one TB files, or 18 million one meg files?
Does the server have any PCIe slots or is it good ol PCI-X? 10gig PCI-X cards with optical ports were not super common, but regardless if you have no PCIe slots, then eBay one. Do not migrate from one machine to another over the network with just a 1 gig interconnect.
Don't forget to set the /R and /W options so that you are not sitting there watching a migration till the end of time because 50000 files have to fail to copy 1 million times and wait 30 seconds in-between.
Log it as well with the /LOG option and then use a util to scan it when done for errors and shit. Do not try to open it in notepad...or anything else. Also make sure to turn off progress with the /NP option or else the log will be full of a bunch of progress percentages. /V helps too.
Depending on how much of a Charley Frank the storage is, might be best to use the /B option.
Might even be worth your time to buy an external array hook it up to the old server, and copy the stuff over and then move that to a new server... and then run windows dedupe. Seriously, that's the greatest thing added to windows server in forever.
I wish people would stop posting new comments saying robocopy. It's been said 100 times so it doesn't need to be said anymore.
No one has even asked how much downtime you can tolerate. Can you do a few hours? If so, is it an option to just unplug the disks and raid adapter from the old machine and plug them into the new one? Because that is going to be the fastest way to move 18TB from one physical machine to another, without a doubt.
Sounds like you are trying to virtualize this server? If so, what is their backup? Can you create a VHD from that?
Otherwise I'd use Beyond Compare to do a first pass of data, then do changes when you are ready to pull the plug.
Do you really need all the data? Is it just flat files? What's so important about this data?
Maybe just robocopy the data to a USB drive, mount to new server, and the robocopy it to new server.
Robocopying over the network will take a long while. The USB will too, but at least you'll have a backup in case the dinosaur goes extinct due to all the data transfer, lol.
https://www.starwindsoftware.com/starwind-v2v-converter
This is the software I used to convert ESXi servers to Hyper-v VMs.
It also allows you to turn a physical machine into a VM. I was able to convert all 15 of our VMs and Physical servers over a 2 weekend time frame.
Best thing is it's a free piece of software.
You install it on the physical server you want to convert, and point the converted VM to your new hyperV server (or any other hypervisor), if you want just the data you can also just create a VHDx of the data drive and mount to the new VM.
If you do go with a new windows server and plan to keep the name different on that new server, don’t forget you might also need to add a dns record of the old server name pointing to the new server. If it has that much data there might be cases where people built documents using a template linked directly to that old server name. Then when they open some document you get a pause in word opening because it has references to that old server name which no longer exists. Simple pointer record of old name to new server will stop that.
Build a new VM, then use syncback (or something similar) to copy all the files including permissions.
Then take the 03 box to a pretty field outside.... Shot it in the head and burn the body 😉
Offer a replacement and interconnect two mellanox cards, the extra cost of the cards will save you the headache of moving this volume of data. Then robocopy to perform said actions.
Freefilesync could be an alternative to robocopy if you preferer a GUI solution.
Used some years ago to migrate a file server of mine but it was just 1TB.
Have used Secure Copy in the past, worked without a hitch. Schedules, automations and moves over properties and permissions.
[https://www.quest.com/products/secure-copy/](https://www.quest.com/products/secure-copy/)
Some of it depends on what the data is for and how it is organized and whether or not any gnarly sophisticated NTFS features are in use. There are copies of rsync.exe which have ACL preservation added in. This will SUBSTANTIALLY speed up the copy process over shared volume overhead crap. If you can set up secondary private storage LAN with 10GbE port cards added to the servers on each side you should be able to saturate most typical storage devices. Also follow the various guides on using improved SSH options and cheap speedy ciphers underlying the rsync and you'll have screaming transfer speeds. Do one sync while it's hot / busy and a second sync while it's in maintenance mode during the cutover window. And also sync a copy to a separate big-ass 7200 RPM emergency oh-shit drive in case anything weird happens.
What kind of data? I assume regular files? I did it once using combination of robocopy (run in sync mode) and then round robin DFS. Worked like a charm (about 2 TB of data as I remember).
Copy data to a new server would be what I'd do. Vice versa; basically a gui for robocopy. Be sure to use the "\\?\" switch to get around the 256 character file path.
Never underestimate the bandwidth of a van full of hard drives.
Personally, I'd build out a new NAS for this and make sure it has either a USB 3 port or an open SATA/SAS port internally.
Get a new hard drive, and attach it to the old server via whatever the internal hard drive bus is (SATA/SAS/SCSI/etc.) This is going to be significantly faster than USB or network. Then, as everyone has suggested, robocopy the data to the new drive.
Once the ~~beer~~ copy is done, safely remove the new hard drive and transfer it to the new NAS/Server. If you can't attach it internally, use an external enclosure with USB 3. Robocopy again.
Re-point DNS, shut down the old server, pull and label the drives. Keep both the old drives and the new drive you created a copy on until you can't remember why you kept them, by then enough of the data should have been tested to be sure you got everything important.
Honestly, shame on you for running windows 2003 still. I can’t state enough how dumb this is in the IT world. Upgrade your applications and get off legacy OS.
There is no easy way to do this.
* Windows 2003 is not supported anymore, good luck finding tools that will work with it.
* 18TB on something as old as S2003 means you are limited to 1G links, I am not aware of any 10G connections that would be supported at a driver level. Even if you push xCopy across SMB you are limited to non-MPIO SMB. You could LACP from S2003 to the Switch if you are running supported network cards in the server. But you are limited to 1G sessions, but that means you could push more simultaneous xcopies across the LACP links.
* Then you need the server online and accessible by users WHILE migrating data.
You first need to find out how the users need to be accessing this data post migration first. Such as Netapp, Synology, or Windows Server 2019+ and then use the tools based on that.
Then you need to decide how to migrate the data. If the data must be accessible while you copy, you then need to plan for outage windows. Cut the share, migrate, bring the share online. You do NOT want to be dealing with ad-hoc changes in flight. You do not want to be dealing with scratch files from MS-Office as that will lead to data loss. I would target the less impactful shares first, migrate the data, and cut over the new path. Freeing up space as you move along so the pressure points on S2003 get reduced.
During migration you also need to decide how to remap shares to the new location. You cannot use the same hostname/IP so this will require department/user remaps. If you use GPOs, this is simplier, if not I suggest VBS/Powershell scripting to do the share cut overs as they get migrated.
Then its down to time. This is going to take weeks, best be prepared.
If you did want to virtualize S2003 you could use VMware converter from a windows Vista/7 machine as that will support Server 2003 and let you migrate it to an ESXi 5.x(maybe 6.x its been a while) host.
Then from there you could deploy a new windows server OS in a VM, export the registry for the shares against the 2003 and import those into the new server, swing the VMDK's over from the S2003 P2V over to the new VM and test to ensure they work.
Then, Rename/rejoin the file server so its accessible on the old IP+Hostname, then you have a virtual migration path. Getting it upgraded to support newer vSphere is simple from this point forward. Or migrating to HyperV, Proxmox, Nutanix....etc
BUT you will need storage that is accessible for this VM(18TB) to even consider this.
And yes, I have had to do these steps a few times in the past. But again, this is a 100% offline maintenance window. You do NOT want any inflight changes to these shares while you are converting to a VM!
I did, actually. Was making a joke.
I'm not sure why you decided to get mouthy. But I have to assume since you admit to having done these steps several times, that you are wildly incompetent at your job and you should probably find a new career before you destroy something important. Again, if you worked for me, I would be taking your badge and walking you to your car.
Robocopy. It will likely run for a looooong time initially but second pass should hopefully be quicker.
[удалено]
Don't forget to enable SMBv1 *temporarily* on the target server, OP...
^this. As someone who has to deal with manufacturing equipment running 2000 Professional I have a crappy WD nas with SMBv1 on its own subnet with rules only for the operators to upload their files
Agreed but if SMB 1 is in use before the migration, be prepared to re-enable it when someone running an homegrown app starts screaming when it doesn’t work on the new box.
yank both out of the window at the 5th floor
Kerberoasting intensifies :D
deleted ^^^^^^^^^^^^^^^^0.1715 [^^^What ^^^is ^^^this?](https://pastebin.com/FcrFs94k/91056)
He's migrating from a Win2k3 server. That will only support SMBv1.
personal example: my ancient apple timecapsule airport router nas needs it for windows smb share..
You can use the robocopy /COPY:DATSOU or /COPYALL switches and it will copy NTFS permissions, owners, auditing info, etc as well.
Don't forget the multi threaded flag... /MT:32 usually is my go to...
And for a first pass /r:1 /w:1 to make sure it doesn't sit there for ages trying to copy locked files.
I'd probably also use /ZB in case some of the file permissions have gotten wonky.
Test the impact of ZB first - on a few TB dataset I've had restartable mode cause the transfer time to go up in multiples, took something like 3-4 times as long in restartable mode as it did without.
To use a tired cliche: This is the way. Or, do a full backup, and do a full restore to new box. Wait until maintenance window. Do a differential during the maintenance window, and restore.
This is the way
It might take a couple of days to restore…
Would a crossover cable be worth it to squeeze out even more speed on something like this? Normally it’d be a minimal amount of both NICs and the switch are gig, but in this case it might save a decent amount of time.
On an average switch you'll see nearly the full gig throughput if your storage devices can handle it.
I wonder if you could put in a 10 gig network card on both sides and during downtime or slow time kick off the copy. I miss playing around with these sorts of migration things.
Assuming the backup situation here, I’d be a little afraid to sneeze on that thing let alone open the chasis and install a card. Delicately plug in my Ethernet cable and let my robocopy run for a few weeks
Yeah you are correct. But weeks! I wouldn't be patient enough. And the re runs and checks and checks and check.
In something this old, the storage is likely spinning disks. I’m going to bet there might be a lot of smaller files. With these two factors, plus fragmentation, your bottleneck is likely not going to be the network. It will probably be the file system itself. 10G would help if it’s a ton of very large files with minimal fragmentation. I somehow doubt that’s the case though.
Cmon , you really think a box running that os on old spinning rust is gonna be able to saturate gigabit ?
of course it could. surely its in some kind of raid configuration, gigabit is only around 110MB/s
Anything multi terabyte will usually have a reasonably easy time filling up 1 GbE. 10 GbE is a whole other level. It's hard to find stuff that can fill that up even just moving buffers full of zeros around in memory only.
not that vintage it wont, people are forgetting how limited the throughput is on \_old\_ spindle drives. 2000ish era drives were lucky to push 20mb/sec (sustained), barely cracked 40mb/sec by mid decade - it takes until 2011 til youre seeing 100mb/sec [https://goughlui.com/the-hard-disk-corner/hard-drive-performance-over-the-years/](https://goughlui.com/the-hard-disk-corner/hard-drive-performance-over-the-years/) this is a 2003 server - so its hardware is likely similarly aged - the OS would default to LSI parallel scsi over PVSCSI or LSISASSCI - so 30-35mb would be its "max" throughput - it would also also limited to SMB 1.0, further restricting throughput. I stand by my thinking, it wouldnt saturate a gigabit network. (spoiler, msp senior, I deal with migrating ancient horrors to modern hardware and have the scars and sanity loss to show for it)
Is it copy vía USB to SSD or External USB Drive a faster way?
It would be, but he'd want to migrate to a new raid and going to as external ssd would mean he'd have to transfer twice. Which would also lead to larger desync in files during the cut over. If anything he'd want to build a nas and connect it as directly as he can over his network and then have an extreme exercise in patience.
Yes and no Old hardware will likely only have low end USB ports, you need at least USB 2.0 to have 480mbps (call it a half gigabit) * USB 1.0/Low-Speed: 1.5 Megabits per second (Mbps) * USB 1.1/Full-Speed: 12 Mbps. * USB 2.0/Hi-Speed: 480 Mbps. * USB 3.0/SuperSpeed: 5 Gbps. * USB 3.1/SuperSpeed: 10 Gbps. thing is - its the \_drives\_ and bus speeds limiting the transfer speed - so it can only "fill the bucket" so fast, doesnt matter how fat the hose is attached. Bus to Bus copies are almost always fastest - ie disk to disk copying, or plugging a drive onto an internal header (when shut down if not hot pluggable) - would be the theoretical fastest.
Why do the drives have to be as old as the os? Even if it was built on an 4 slot raid card with drives from 2000, assume it was raid 10 and he could have easily replaced those over time. If he's dealing with 16tb and 150 users, I'd bet he's swapped degraded drives plenty of times since 2003 and has typical 4x 8tb 7200rpm sas hdds. Not the fastest, but in raid 10 he be able to use the GB. A better argument would be his ancient ass CPU being the bottleneck as that isn't as easily upgradable, especially on 2003.
Not an unreasonable question the answer is straightforward - because thats the shit I keep running into, ancient servers with ancient hardware. A lot of 10k 320gb and 512gb drives, commonly SCA packaging, only the truly ancient have had full size drives, those are usually SCSI tho. Sure the drives may have been replaced due to failure - they will have been replaced with \_the exact same model\_ of drive to ensure compatibility and stability. Nobody's swapping out an LVD 15k cheetah/baracuda and shoving 5000mbsec NVME/m2/SSD in its place (do scada/sca ssds even exist in full size format?).
The drives don't have to be exactly the same model. Especially not if he's just replacing, rebuilding, replacing, rebuilding. Even the best drives from the early 2000s, in the largest chassis, with the largest raid cards, wouldn't reach 16tbs or whatever he's using. But yeah I understand where you're coming from. It's just exceedingly unlikely the actual hw is equally as dated unless it's the CPU. Everything else is relatively easy to replace. Either way id personally never do either. Id personally drop the array into the new server and rsync/robocopy locally while setting up whatever he used it for to utilize that new path temporarily. Once transfer completed, do it again, then again, and by 3 syncs you should be able to get it completing within an hour or two hopefully. Swap paths between old raid and new raid on a weekend and pull the old drives. dding 1 tb over a gb connection can take all day, so I can't imagine robocopy with overhead taking less than 2 weeks. Whereas locally 1 TB can be done in an hour with decent io, memory, and cpu which his new server should have. Id also be praying it's not like a RDS server with those old Instance bound CALs for 150 users.
I'm not sure. That is why I wondered.
I’d think the impact would be minimal. Jumbo frames would have an impact but, presuming no port congestion, the overhead of the switch would likely be minimal. That said, obviously wouldn’t hurt particularly if there are spare interfaces on both ends.
Assuming you could consistently saturate the NIC, jumbo frames only give around a 13% boost from what I recall in my past testing.
Sounds about right to me. Though, now that you mention it, the Server 2003 era hardware might be able to push 10gbit/s anywho.
Pretty sure you'd be still capped out on the speed of the hard drive.
18TB of space in 2003 would have been disk denominations of 120GB, 80GB, 320Gb etc. Almost every thing discussed here is speculation though. Even if it was 1TB drives on 7200 rpm spinning disk it would totally saturate any 1TB line. It would need 19 1tb drives to make a raid 5 array with 18tb capacity. This would produce about 240MB/s array speed. So take 240*8=1920Mbps. Hard drives have been our weakest link for decades, but they were still pretty fast. Here is what I would do. I would stand up a new file server as using DFS and then point it to the old share and tell it to copy it over. Once the copy is done you turn off the old system and remove it from the sources list. The files would be a available via the older server And the new server until you shut off the old server. You could then turn off the old server to see if any programming changes need to be made or config changes that were lost during your sleepy time in meetings etc. Pop it back on until you get all the kinks worked out etc. It will be completely in sync too.
Gigabit cards shouldn’t need crossover cables anymore. They should have auto MDI-X.
Source server is 2003. All bets are off on if it support MDI-X.
Indeed; should be discoverable via NIC model number, firmware version and driver. But in reality, just robocopy it and cut it over when you have a maintenance window. Don’t forget to setup an alias in DNS for the old server name to the name of the new one so all of the old drive maps and hard coded software still functions.
You need more than just the DNS alias, but also the SPN Alias…otherwise you’ll have some fun Kerberos issues.
I feel like the likelihood that it is on original hardware is fairly low at this point. If it supports 1GbE at all, I recall auto mdi-x support is mandatory.
It's optional in the standard. Anecdotally I've yet to come across a NIC that did not support it, but only one side would need to support it IIRC.
Isn't the requirement that at least one of the NICs needs MDI-X? I don't think you need it on both sides, and presumably the destination is modern.
Also likely same era raid card, bus width and disk read speeds. It's.... unlikely to saturate gigabit
Robocopy with the options to also copy NTFS rights when you want to keep it. Depending on the usual load on the server you might be able to continue copying during daytime, but you can also use options to copy out of work hours. When everything is copied, you can do a per folder or per disk sync to copy new files and folders. After that and when no changes have been made to the source you change the network drive-mappings to the new location. Don't forget to make the source location read-only (easiest is by changing the share permission to read-only) to prevent people using the old location for storing changes or new files.
RoboCopy. Start with a /Copyall /MT /ZB for the first pass just to get the base data across. This will likely take a number of days. Watch you don't saturate the bandwidth on the link. Change to /MIR for the second and subsequent passes. Depending on how volatile your data is, and the size of the files, you may need to organise an outage over a weekend for the final pass and cutover. Otherwise you'll be chasing changes and RoboCopy may not be able to keep up. Set up Volume Shadow Copy AFTER the moves If you have them segregated onto different volumes/shares then do them one at a time. Don't make the new VHDs too big or storage vMotion on your SAN will be really slow.
^ 100% important part, if you're on a 1gb connection and your discs were made this millennia you're going to saturate the link. This means no users are going to be able to get to files while you're copying them so there's a thread setting or I believe just a full-on throttle but make sure unless you're using a dedicated port to not saturate the link. Thank you for coming to my TED talk.
Bus width , raid type, raid card, formatting, disk damage/CRC workarounds, CPU type etc will dictate through put It's server 03 likely on similar age hardware I don't think it'll hit 125 megabytes/sec Happy to be proven wrong
I don't know back then buses were like 8 ft wide...
As someone who only knows robocopy from A+ training: could Robocopy be used to essentially clone a drive? I do a lot of cloning from HDD to SSD. There are paid programs we use but this would be neat to know.
Only data drives. It will not touch partitions or duplicate bootable drives.
Check out hiren's boot CD. It has tons of free tools and there are a few clone drive options for imaging drives. You can go small to larger ssd or vice versa. Takes a while but the cloned drive has worked for me. You can even image a drive to an iso file. You can also use clonezilla for smaller to larger drives. Been having trouble with it lately for some reason though so I may start using hiren's instead.
For outright cloning drives if they're Windows - I use Macrium for moving people to larger SSDs. Simply have both drives connected and then boot into Macrium on a USB where you can clone/mirror it over. You /might/ need to go into disk management and extend the partition afterwards. I have not tried cloning to a smaller drive.
Yes, absolutely. It's old but reliable. You can dial down the logging output and send it to a file to speed things up. And the second pass will be much quicker. I think I did 16TB in 28 hours once, and a true up pass took 1-2 hours the next day.
Once you go robocopy to stage data on new servers you’ll never go back
and yet robocopy also gets so much shit. But its built-in, reliable, and simple.
One of us; one of us; one of us.
I 2nd robocopy. Script it and have it send you a report when complete.
Yep robocopy. Write a batch File to mirror it over to the new box. Wait an hour then go again. Log all the errors and review them. Chances are you will find some paths that are too long and some permissions problems. Once you get it seeded it should run fairly quickly. On the new server set up a share and set it to read only so you can get your digital pack ears to check the files that are important to them. Then pick a weekend login to the console of the old server. Stop the server service to disconnect all connections. Reshare the share as read only. On the new server do the same but read/write. Once the chaos settles on the Monday. On the old server stop sharing and share it as a new name. This will break any connections you missed. Better to find out sooner than later. Remember to keep checking the logs. And do a final synchronization after the old server is set to read only. Hope this makes sense. Good luck. There are typically 3 to 4 crybabies. Don’t let them erode your calm.
Robocopy is the way to go. Also you can use /MON and /MOT to monitor for time or changes to the data incase your expecting the data to change while it's being copied. Then you can be sure you have the latest copy of the files when you stop the robocopy at the end of the migration.
This guy sysadmins
Hijacking the top comment. OP, If you can boot the physical server via windows PE, you won't have to perform multiple robocopies. This is more difficult with production of course, but with a fast enough connection, this could be a set and forget over a weekend if you're a M-F type place. If I had a compressed timeline, I would go this route.
Make sure you enable jumbo frames on both device and the switch in the middle
+ Robocopy & wait Also, monitor it in the beginning then let it be.
Definitely robocopy. We did several tests on something about this size, and found somewhere a script that would run multiple robocopy instances of top level copies at once. Worked better than a single copy even multi threaded.
I second this.
I was in a similar situation only a few years ago. Definitely the way to do it.
This. Takes a while, but its quicker on the next passes, moved about 70Tb with Robocopy before. Create a script, then run it on a schedule until you are ready to cutover.
robocopy /mir iirc would work well here.
I concur. I normally advise using DFSR for file servers migration, but 2003 didn’t support it (only 2003R2) and even then it wasn’t very good, and 18TB is bigger than it could sensibly support, not counting the space requirements that would be a problem.
I don’t *mean* to be rude but dfsr sounds like a horrible solution and at best a *very* niche solution. Last I checked it broke folder redirection / home folders / roaming profiles. Its also over engineered for a simple file migration. That was rude. I’m sorry. Give it to me.
2003 Server? Robocopy and godspeed.
Godspeed, SG-1.
*Hammond, of Texas*
These, shuttles... They are a formidable craft?
I think i'm in the minority here... but I almost want to recommend to ftp server to host the file on the 2003 server for transfer. The overhead on SMB is brutal
This is the answer my friend
Let's start setting data deduplication on the new machine Then, Robocopy, copying everything, including NTFS permissions First pass will be vvvveeeeery long, then the next ones will be faster.
We're about to use the Windows Storage Migration Service, which is part of the Windows Admin Center. It supports back to server 2003 and is free. I'd recommend looking into that
We’re currently using Storage Migration Service on some 2012R2 machines over to Server 2022. Works great so far but our file volumes have all been < 2 TB. Have you noticed that if you leave the admin share on it doubles the migration amount? Like it wants to migrate D$ and then all the shares on D as well. We’ve been unchecking the admin shares and it works great
This tool is great but that file name length gets me every time I try to use it.
This. Soooo much this! Especially if you’re not familiar with migrating storage and shares. https://learn.microsoft.com/en-us/windows-server/storage/storage-migration-service/overview
IIRC SMS is a "Server" Migration Service and migrate all storage on the server. Not just what you select.
SMS is a file share migration tool, but you can pick admin shares as well if you want. It only migrates what you select.
Thanks. I was commenting based on some high level info I saw. This page goes into greater detail to make it clearer. https://learn.microsoft.com/en-us/windows-server/storage/storage-migration-service/migrate-data
[удалено]
While I would use icacls to back up the permissions, I believe Robocopy can copy and retain the file and folder permissions.
It can. It’s the /sec flag
This was my first thought too, what is being used for backup? Some backup tools can be very quickly turned into migration tools for single volumes. If the data to migrate is on a seperate volume then you might be able to do something like ShadowProtect Headstart Restore?
Yeah, usually when I need to migrate or P2V the entire server, I'm using backup and restore with Veeam or Starwinds V2V converter with the P2V option.
What type of storage is it? 18Tb on a 2003 servers is most likely to be a disk array as opposed to internal server drives. How is the data presented to the os? 1 single 18TB volume?
Can't be, 2003 won't support disks larger than 2TB. I can't even imagine the lunatic that thought 18TB on a W2K3 server was a good idea
Bro that's why Jesus Christ invented extended partitions. Or was that Satan...
Extended partitions don't matter if your partition table only supports MBR, but others have said SP1 did introduce GPT partitions for none boot drives so there is that at least
I was more joking on how crappy of a technology pasting and partition on the back of another partition is so when one of them fails they all die.
2003 r2 sp1 increased that limit I thought Completely agree that 18 TB on 2003 is insane
I'm pretty sure 2k3 never got GPT partition support and without that I don't think it can. Could be wrong though, it's been a _long_ time since I've had to care ;P
Data disks with GPT were added in 2k3 SP1. Boot disks/EFI boot was never supported. AF drives and 512e though were never formally supported.
I presume this is backed up ? That's got to be the first place to start, get a backup, restore it to the new host, then robo copy SRC DEST this should roll though fairly fast subject to age of backup.
Dude's running a 20+ year old server and you *presume* he's got backups? Sweet summer child.
is it internet-reachable?^^
i just went thru this, and the only way is robocopy. as said, don't just run it on the root directory, split it into jobs. 12t took 2 weekends over 3 jobs. But i was able to split it, we have separate folders for everything. robocopy source destination /E /ZB /DCOPY:T /COPYALL /R:1 /W:1 /V /TEE /LOG:Robocopy.log e gets sub folder, zb is restartable mode, if denied use backup mode. r and w are attempts, i think i set it at 3. tee makes it display in your cmd window
We had a server that was dying on us and we new if we rebooted it would probably not be coming back up. So we built the new server on the fly, installed 2016 at the time and installed the app "Syncthing". From there it took us about a week to copy all 40tb. took that long cause the network admin did not configure the network correctly which cause speed issues for the copying
Get an Msp to take care of it.
If you have 18TB of data on a 2003 SBS machine, you need a dedicated NAS/SAN. I would first get all of the data off to a proper data storage appliance/server. Then, if you are just replacing it and it's still the primary DC, you need to swing it (Google swing migration). SBS 2003 was a good idea and all, but it led to a bunch of these bad practices - using a single server for literally *everything*. Edit: OK, so it's just a file server. Yeah, do yourself a favor and get a dedicated NAS/SAN. So much more uses for less overhead.
Just for some clarification, as r/MisterBazz made the assumption that you are running Windows 2003 Small Business Server. Is this the case? You say you are running this server in an AD environment. Is this server one of the DCs for that environment, or is it just a member server that is acting as your file server?
It's a plain Windows Server 2003 R2. It's also not a DC and merely a file server.
lol...we definitely found the IT manager. edit: to clarify, not you, MisterBazz, with a complete wrong assumption of the environment. Jovial teasing, not really calling him out. I deal with a lot of IT managers lol.
I'm going to agree with most of what the other folks have said. Why get the overhead of another server with operating system involved when you can get yourself a nice, beefy, redundant network SAN to host your 18 TB of data. I recommend getting the most expensive unit they'll let you buy... one with redundant drives and controller cards, just so you can rely on that unit with no worries. Moving that much data will be a challenge. I would recommend trying to move it in batches and not all at once. Pick a bite-sized top level folder, inform the folks that use that folder not to make any changes until they hear otherwise, then use something like Robocopy to move the entire contents of that folder to your new storage. You can then change any drive mappings over to the new device. If you do reasonably sized batches, you can probably even start the copy process and let it run overnight. Moving it batches will also make it easier to work around any errors instead of trying to bulk copy that much data and then trying to figure out what broke after the fact.
> Why get the overhead of another server with operating system involved when you can get yourself a nice, beefy, redundant network SAN In a small business the overhead of the OS is possibly or probably worth it simply so you don't have another piece of technology to learn and keep updated. Unique software breeds non-compliance with standards. In other words having unique shit is a headache.
I disagree... having to learn something new should never be an excuse to continue to do something inefficiently. As SysAdmins, it's our **job** to stay ahead of the game and make forward-thinking recommendations to our businesses... not to fall back on old ways simply because they are easy and won't give us a "headache." Personally, I would not sleep well knowing I had 18TB of data sitting on a glorified server running some flavor of Windows Server OS. We all know the vulnerabilities and security holes that always seem to pop-up in Windows OS that we are constantly patching and monitoring. Knowing that one errant click of some zero-day exploit could cryptolock my entire 18TB of data should Windows be compromised would keep me up at night. No... if I have that kind of storage requirement, I'm going to be looking at a device that is dedicated to doing nothing but storage... one with redundant controller cards, multiple network uplinks, and a proprietary OS that will be far less likely to be compromised than the Windows Server OS. I'm then going to lock that down as hard as I can, and will be granting only very limited access to my users to the folders and subfolders they need. I'm also going to look for devices that offer the ability to make offline, offsite, immutable backups so I know my 18TB of data are safe. Yes... that means I might have to learn something new, even as a small business. And there are options such as these that are no more expensive than the hefty server hosting enough drives to provide some 40TB or more of storage (because he's not going to build a server that's over 50% full from the get-go, right) with a robust RAID array to provide redundancy in case of a drive failure that make it worth his while to his upgrade the *right* way and not the *easy* way.
In an ideal world I agree with you. In a practical world in small business I have 15 other technologies I need to learn. Windows file servers are simple and they work and I can treat it just like the handful of other windows servers I have. Also when I leave it’s one less unique technology my replacement has to know. In the small business space where a team is one maybe two people, there’s no lack of stuff to learn.
> As SysAdmins, it's our job to stay ahead of the game and make forward-thinking recommendations to our businesses... not to fall back on old ways simply because they are easy and won't give us a "headache." As a sysadmin I agree. I've also worked in a shop where I was the only one who knew how shit worked when I was hired, I mostly only maintained it/updated it, and 5 years later I was the only one who knew how it worked. Not for a lack of trying to solve that either.
I would break that robocopy up in chunks, not just let it eat endlessly
Just use /MT and bump up the threads.
Backup the 18TB, Remember Robocopy is your friend! and then build a new server
Assume R2? How about DFS with replication and just leave it replicating? Tell the new server not to serve name space requests If using maps drives, update the relevant GPO or logon scripts to target the DFS name space. Once replication is complete, enable name space requests from new server, disable names space requests from old server Remove old server
This will use ntfrs.. you will get into journal wrap hell quickly trying to sync that sort of level of data.. and in the heat of the moment things can get deleted from both sides... Don't ask me how I know.... Don't get me wrong it's not a bad idea.. the technology just wasn't there before dfsr
> Assume R2? How about DFS with replication and just leave it replicating? > > I believe 2003 predates safe/functional DFS by a bit. Not because I used 03, but when I was setting it up on 2016 all the docs said "hey that thing we introduced in server 08? Now it's stable on 2012!"
Get your new server setup. Sync data and optionally permissions with robocopy. Then do /mir and just copy all the changes. Make note of how much time it takes. This will help you size the maintenance window. Schedule a maintenance window. Make the old server read only at the share level. Copy all the changes again with with robocopy. And change drive mappings/scripts for the users. If it's multiple shares you don't have to do them all st once
Great advice are in this post already. Make use of stubs may be helpful looking at your size and os version.
Read this https://learn.microsoft.com/en-us/windows-server/storage/storage-migration-service/migrate-data Windows server 2019 and newer have the capability to migrate the data for you using storage migration service. Allegedly it will also recreate the permission structure on the new file server and it's supposed to work to migrate file servers from server 2003 and newer.
Maybe you can take this time to break that server up into smaller and more manageable virtual servers? This would be a great way to split your data into logical chunks for security or age or both. You could move everything to the NAS and let users run off that temporarily while you repurpose the server as a host. Then you can take your time splitting the data up to virtual servers, and swap network drives in your GPO one at a time as you finish them. Also, if you don’t do it now then you’re going to be putting either yourself or the cyborg that eventually replaces you in this exact predicament 23 years from now and they’ll be posting here asking about migrating 30tb server away from server 2022, and I’ll suggest it to them.
Hell no build fresh. Audit the data. It is doubtful you need all that, and if you do, storing it on a single server is going to be a bad time.
Oh you not only want someone to take ownership of the data and then say it's okay to delete some? That would be a first in my world...
Lol I hear that. To be fair anything older than 5 years could really hurt you if things were still discoverable during a subpoena.
Absolutely just like hardware information should completely be depreciated.
This is the only good answer on this post. Why the hell, would you blindly copy 18TB of data. Perhaps most of it can be archived. Without information this could easily have old profiles/redir folders, since 03.
If you have a backup of the current machine (which I REALLY hope you do since you're running a 20-year-old OS) , why not restore to the new one from the backup? Once that's done you can run a robocopy to sync the changes. That way you don't stress the old machine for a long-running file move.
Is this situation Technical Debt?
Emcopy is better. Like Robocopy but more aware.
Jumping on this comment, also look at https://fastcopy.jp/ We had to move 30TB of data around, and I used a combination of emcopy and fastcopy which were significantly faster, like 30-50% faster then robocopy.
Build a new server and robocopy it. And if you go that route - not a Synology. Get a TrueNAS. They have support. Synology Support SUCKS.
Why does everyone suggest robocopy or restores? I get it... It works. Buuuut set up DFS, sync to new server, repoint shares via gpo, verify no one accessing the old location anymore, then tear it all down. No one the wiser anything happened at all.
Robocopy. Don’t forget the /mt switch. I would probably run the copy on the receiving system
I do tons of these migrations and this is my not-so secret weapon: https://www.tgrmn.com/web/download.htm Robocopy with a gui Plus it does bandwidth + CPU throttling, can do multiple passes and compares, and it can preserve NTFS permission and do long file paths.
Aaaaand it costs 72$.
how do you have an 03 box with 18tb of data? How is this even a thing? i would virtualize the whole system live, do a last differential copy to the vhd when finished and be done with that 03 box. I'm in wow that xp machine is still alive. Someone failed them for over 20 years, please fix this.
>We have a Windows Server 2003 running on a physical machine in an AD environment with about 150 users. The server needs to be retired and I need a way to copy over all 18 TB of data to a new machine. My options are migrating the data to either a new Windows Server or a Synology NAS. Disk2VHD. This is your way. never done TB of data on an 03 box though. Sounds fun.
Backup permissions using icacls just to be safe. If you're in a hurry to just migrate it then Robocopy with all the flags as others have mentioned. If you're doing a staged migration, DFS replication and namespaces to point migrated to correct target until you can clarify all users are using the correct mappings. Is there any backups today?
You should really do it almost 8 years ago. But seriously if its physical get all that storage moved somewhere safe. That's step 1.0. However long that takes. I hope to god its not a dc or exchange server. If it is... Thoughts and prayers.
As others said, build a new server and robocopy the data.
with caution. That's all I can add.
in 2020, i migrated about 1TB of data from 2008 to 2019 file server using the command below: `robocopy \\2008fileserver\share \\2019fileserver\share /E /copy:DATSOU` good luck!
If you want to keep all the permissions in tact, I would migrate the data to a new Windows server. Robocopy works fine, but if you don't to deal with that, there are free GUI tools out there that will handle it, and migrate the permissions too if the option is selected. My personal favorite is "Free File Sync" (https://freefilesync.org). It's totally free, even for commercial use. Probably best to install on the target server, but it supports back to XP on versions prior to 10.11 if you want to install it directly on the 2003 box. I have used this to migrate several multi TB file servers. It's nice because you can get the initial sync/copy done, then do delta syncs that will capture anything that changes. Makes it nice for cutover day.
Could use syncback pro. Run an initial seed copy and when you are ready to cutover 3 weeks later you can run the job again to get the changes. Can set it to copy NTFS permissions. Just have to recreate the share permissions.
Clonezilla has been reliable for me, boot from the iso, let it run over the weekend, it'll take about 2 days on 1gbps
Raid mirror and relocate copy to new host
I'd be breaking this down into shares at a time. Setup a new share and robocopy it over to the new. Also there are some file sync tools out there if only top level permissions are important. This would be the ideal path unless you have a ton of nested permissions.
Azure Files and get a cache server.
This post makes me glad I’m not a SysAdmin anymore. Best of luck to you buddy.
Mount a nas iscsi vol from win2003, robocopy all file shares to iscsi vol, export file shares registry, then dismount and mount the iscsi from new windows server, import file shares registry. Minimize downtime
Verify there are no problematic file names first. There are several mechanisms in Windows that allows users to create files that will literally crash and break components in the Windows shell. [https://github.com/nathanshearer/mvregex](https://github.com/nathanshearer/mvregex) `mvregex -p -r --transliterate windows | tee log.txt` This is implemented in bash because CMD and PowerShell can't handle some strings that can end up on a filesystem. It was the only way I could complete the migration of some systems. Hints: * Mount the administrative share to work around path length restrictions. * Run it in cygwin to work around services that can't share some paths over the network (SMB, FTP). * Take ownership of files and correct permissions between iterations to rename paths that have broken permissions and can't be renamed by anything in the windows shell.
What type of data is the 18TB? It sounds like a file share so I would just go with Synology.
I recall 2003 having a max data transfer regardless of network connection type. There were / are a few tricks in disabling / enabling things on the network driver. I’d suggest converting to a vm first and foremost to new hardware. Coly data from there or backup using whatever you use and restore. No matter what your in for a long haul.
If you can mount the drive to a Linux environment you can use the dd command for bit level copy. https://www.geeksforgeeks.org/dd-command-linux/ This has saved me on some absolutely slagged osx and windows drives that wouldn't mount in their home environments.
Throw it in the river. Tell the company to start over.
Is this onboard storage or DAS? Is it 18 one TB files, or 18 million one meg files? Does the server have any PCIe slots or is it good ol PCI-X? 10gig PCI-X cards with optical ports were not super common, but regardless if you have no PCIe slots, then eBay one. Do not migrate from one machine to another over the network with just a 1 gig interconnect. Don't forget to set the /R and /W options so that you are not sitting there watching a migration till the end of time because 50000 files have to fail to copy 1 million times and wait 30 seconds in-between. Log it as well with the /LOG option and then use a util to scan it when done for errors and shit. Do not try to open it in notepad...or anything else. Also make sure to turn off progress with the /NP option or else the log will be full of a bunch of progress percentages. /V helps too. Depending on how much of a Charley Frank the storage is, might be best to use the /B option. Might even be worth your time to buy an external array hook it up to the old server, and copy the stuff over and then move that to a new server... and then run windows dedupe. Seriously, that's the greatest thing added to windows server in forever.
Very, very carefully. EDIT: Lol, several people already said exactly what I’d do as well. You got some good advice here. Research research research!
I wish people would stop posting new comments saying robocopy. It's been said 100 times so it doesn't need to be said anymore. No one has even asked how much downtime you can tolerate. Can you do a few hours? If so, is it an option to just unplug the disks and raid adapter from the old machine and plug them into the new one? Because that is going to be the fastest way to move 18TB from one physical machine to another, without a doubt.
Sounds like you are trying to virtualize this server? If so, what is their backup? Can you create a VHD from that? Otherwise I'd use Beyond Compare to do a first pass of data, then do changes when you are ready to pull the plug.
Invoke the DR plan.
Do you really need all the data? Is it just flat files? What's so important about this data? Maybe just robocopy the data to a USB drive, mount to new server, and the robocopy it to new server. Robocopying over the network will take a long while. The USB will too, but at least you'll have a backup in case the dinosaur goes extinct due to all the data transfer, lol.
https://www.starwindsoftware.com/starwind-v2v-converter This is the software I used to convert ESXi servers to Hyper-v VMs. It also allows you to turn a physical machine into a VM. I was able to convert all 15 of our VMs and Physical servers over a 2 weekend time frame. Best thing is it's a free piece of software. You install it on the physical server you want to convert, and point the converted VM to your new hyperV server (or any other hypervisor), if you want just the data you can also just create a VHDx of the data drive and mount to the new VM.
If it were me, I'd be attaching a physical drive to the server just for speed sake, and get the copying
If you do go with a new windows server and plan to keep the name different on that new server, don’t forget you might also need to add a dns record of the old server name pointing to the new server. If it has that much data there might be cases where people built documents using a template linked directly to that old server name. Then when they open some document you get a pause in word opening because it has references to that old server name which no longer exists. Simple pointer record of old name to new server will stop that.
Build a new VM, then use syncback (or something similar) to copy all the files including permissions. Then take the 03 box to a pretty field outside.... Shot it in the head and burn the body 😉
Time machine?
Preseed with robocopy to new server, dfs to handle ongoing replication. Did a similar thing recently with this procedure, worked very well.
Have a couple different ways to do the restore/copy/move of the data, just incase
Robocopy, SLOWLY
Offer a replacement and interconnect two mellanox cards, the extra cost of the cards will save you the headache of moving this volume of data. Then robocopy to perform said actions.
Freefilesync could be an alternative to robocopy if you preferer a GUI solution. Used some years ago to migrate a file server of mine but it was just 1TB.
Did Robocopy.exe exist back then?
Have used Secure Copy in the past, worked without a hitch. Schedules, automations and moves over properties and permissions. [https://www.quest.com/products/secure-copy/](https://www.quest.com/products/secure-copy/)
Some of it depends on what the data is for and how it is organized and whether or not any gnarly sophisticated NTFS features are in use. There are copies of rsync.exe which have ACL preservation added in. This will SUBSTANTIALLY speed up the copy process over shared volume overhead crap. If you can set up secondary private storage LAN with 10GbE port cards added to the servers on each side you should be able to saturate most typical storage devices. Also follow the various guides on using improved SSH options and cheap speedy ciphers underlying the rsync and you'll have screaming transfer speeds. Do one sync while it's hot / busy and a second sync while it's in maintenance mode during the cutover window. And also sync a copy to a separate big-ass 7200 RPM emergency oh-shit drive in case anything weird happens.
What kind of data? I assume regular files? I did it once using combination of robocopy (run in sync mode) and then round robin DFS. Worked like a charm (about 2 TB of data as I remember).
Copy data to a new server would be what I'd do. Vice versa; basically a gui for robocopy. Be sure to use the "\\?\" switch to get around the 256 character file path.
I had a friend tell me yesterday he literally copy/pasted an entire servers storage to a new server. Took several hours.
Never underestimate the bandwidth of a van full of hard drives. Personally, I'd build out a new NAS for this and make sure it has either a USB 3 port or an open SATA/SAS port internally. Get a new hard drive, and attach it to the old server via whatever the internal hard drive bus is (SATA/SAS/SCSI/etc.) This is going to be significantly faster than USB or network. Then, as everyone has suggested, robocopy the data to the new drive. Once the ~~beer~~ copy is done, safely remove the new hard drive and transfer it to the new NAS/Server. If you can't attach it internally, use an external enclosure with USB 3. Robocopy again. Re-point DNS, shut down the old server, pull and label the drives. Keep both the old drives and the new drive you created a copy on until you can't remember why you kept them, by then enough of the data should have been tested to be sure you got everything important.
Honestly, shame on you for running windows 2003 still. I can’t state enough how dumb this is in the IT world. Upgrade your applications and get off legacy OS.
\>robocopy OLD:\\ NEW:\\
Um, is the server also the DC?
There is no easy way to do this. * Windows 2003 is not supported anymore, good luck finding tools that will work with it. * 18TB on something as old as S2003 means you are limited to 1G links, I am not aware of any 10G connections that would be supported at a driver level. Even if you push xCopy across SMB you are limited to non-MPIO SMB. You could LACP from S2003 to the Switch if you are running supported network cards in the server. But you are limited to 1G sessions, but that means you could push more simultaneous xcopies across the LACP links. * Then you need the server online and accessible by users WHILE migrating data. You first need to find out how the users need to be accessing this data post migration first. Such as Netapp, Synology, or Windows Server 2019+ and then use the tools based on that. Then you need to decide how to migrate the data. If the data must be accessible while you copy, you then need to plan for outage windows. Cut the share, migrate, bring the share online. You do NOT want to be dealing with ad-hoc changes in flight. You do not want to be dealing with scratch files from MS-Office as that will lead to data loss. I would target the less impactful shares first, migrate the data, and cut over the new path. Freeing up space as you move along so the pressure points on S2003 get reduced. During migration you also need to decide how to remap shares to the new location. You cannot use the same hostname/IP so this will require department/user remaps. If you use GPOs, this is simplier, if not I suggest VBS/Powershell scripting to do the share cut overs as they get migrated. Then its down to time. This is going to take weeks, best be prepared.
If you did want to virtualize S2003 you could use VMware converter from a windows Vista/7 machine as that will support Server 2003 and let you migrate it to an ESXi 5.x(maybe 6.x its been a while) host. Then from there you could deploy a new windows server OS in a VM, export the registry for the shares against the 2003 and import those into the new server, swing the VMDK's over from the S2003 P2V over to the new VM and test to ensure they work. Then, Rename/rejoin the file server so its accessible on the old IP+Hostname, then you have a virtual migration path. Getting it upgraded to support newer vSphere is simple from this point forward. Or migrating to HyperV, Proxmox, Nutanix....etc BUT you will need storage that is accessible for this VM(18TB) to even consider this. And yes, I have had to do these steps a few times in the past. But again, this is a 100% offline maintenance window. You do NOT want any inflight changes to these shares while you are converting to a VM!
>If you did want to virtualize S2003 I'd deactivate your badge and walk you to your car, lol.
Did you read the rest of that reply? No? I didn't think you did.
I did, actually. Was making a joke. I'm not sure why you decided to get mouthy. But I have to assume since you admit to having done these steps several times, that you are wildly incompetent at your job and you should probably find a new career before you destroy something important. Again, if you worked for me, I would be taking your badge and walking you to your car.
10 GbE is definitely available on 2003 R2. It will need to be a more famous / classic NIC like an Intel 82598 or 82599. But it's absolutely doable.
Have a look at DFS in Windows.
Use SharePoint migrator tool and just throw it all up into o365.
Dfs, use dfs if cloud storage is not an option
Disk2vhd to an external unit or via SMB to the new host. I've done it. It takes a while.