Another long gap between since my last post, what else is new.  I have no idea where the time has gone between working on development projects and testing too many things to list. So to kick off this post here is some interesting personal perspective on security in virtualization.

Now I am more than aware that 99% of the virtualization technology in production today is VMware and that Hyper-V is not at the top of most organizations’ consideration. I hope to change that perspective through this small review of some approaches that NVINT is taking as a small company. I am not here to try and sell Microsoft products (I am more of a Google Fan than anything) but when something is done right for more than one reason I always will like that product.

This will be a multi-part blog post that covers a few topics across Server 2012’s Hyper-V 3.0 to the 2012 System Center product family as they relate to the DFIR world.

Hyper-V

I will start with Hyper-V 3.0 inside server 2012 because there are a lot of great new features that really put it above other hypervisors. The first thing that really stands out as a major feature in 3.0 is that you can fully encrypt your storage clusters and allow access to that Shared SMB cluster over the network. This allows you to separate the Hyper-V boxes hosting the Virtual Machines from the machines hosting the SMB Storage cluster. This allows you to lock down access from only the Hyper-V servers to the storage cluster and monitor what accesses that data directly.

At NVINT I currently work with a Hyper-V 2008 R2 cluster that hosts over one-hundred virtual machines across six HP blade servers that make up the Hyper-V host machines. This Hyper-V cluster hosts the production virtual servers for our clients. I have been testing Hyper-V 3.0 for the past ten months, and I still feel like I have only opened one of the presents Santa left me under the tree. In my experience the stability and redundancy is really now on par with other hypervisors as where our current production 2008 R2 Hyper-V cluster lacks.

Windows Server 2012 features a lot of new capabilities, such as the new cloud networking features that scale a higher level network across WAN connections. It also contains the same powerful security features from the past like ASLR and DEP that are critical for security. I am not going to go into the infrastructure aspects because that has been done by the Microsoft engineers and I have provided the links below.

Check out these links if you want to know more about Server 2012 Hyper-V features.

Windows Server 2012 Hyper-V & Network Card (NIC) Teaming – http://www.aidanfinn.com/?p=12924

Hyper-V Network Virtualization Gateway Architectural Guide – http://technet.microsoft.com/en-us/library/jj618319.aspx

Hyper-V Network Virtualization Overview – http://technet.microsoft.com/library/jj134230.aspx

Network Virtualization technical details – http://technet.microsoft.com/en-US/library/jj134174.aspx

Windows Server 2012 Hyper-V Replica http://www.aidanfinn.com/?p=12147

The major upside to Hyper-V for NVINT is the fact that the Datacenter edition of windows allows you to host unlimited Standard editions of Windows in virtual machines without any extra license fees. (Note: this has limitations but that’s outside the scope of this post)

There is one really big feature of Hyper-V that I want to point out, as I feel there is a huge benefit from an Incident Response/Forensics point of view in terms of what you can do with the system that other hypervisors lack. For starters there are a large number of tools that you can use to analyze the Windows OS in many different ways. One of the best features is the ability to take a memory dump of the host os itself.

I have taken memory captures up to 80GB in the system and used Volatility to analyze those dumps successfully. This allows me to peer into the virtualization platform with more in depth detail that can be critical to understanding what is going on in the environment. Most people use VMware which makes it the most targeted Platform. Advanced techniques can be used to detect if the system is a virtual machine and even exploit out of the system to the host causing problems they were intended to prevent.  This may change in the future, but currently Hyper-V is not the largest target on the market in its virtual machines. I am well aware the Windows OS is targeted by everything because it’s used by almost any originations today. I also feel that VMware currently has risks with USB support, currently however that isn’t a drastic problem yet. Someday however I would guess that it will become a more targeted exploit.

Alongside Server 2012 and the features it brings to the table the system center products allow a great deal of management and monitoring across all typed of devices outside the Microsoft Family.

DPM 2012

I currently work with Data Protection Manager almost every single day, and I have come to enjoy this product for a few good reasons. A major benefit for DPM is that it allows you to actively back up the memory as a scheduled process that Microsoft calls a “System State” backup. It also has standard features with Full-Disk and Incremental backups based on a scheduled format. It does anything from attached SAN storage to Tape backups which allows you to scale it out fairly easily.

At NVINT we maintain 15 days of full disk backups and the “System State” included. This allows me to restore a Virtual Machine to a saved state inside Hyper-V or to a network location much like the features inside VMware consolidated backup. The huge benefit from an analysis standpoint is that Hyper-V stores the system RAM for a virtual machine in two separate files the .bin and the .vsv which can be used with vm2dpm  http://archive.msdn.microsoft.com/vm2dmp to convert them into a crash dump format that is compatible with Volatility or Microsoft Debugger for analysis. In an incident I can go into my Hyper-V server and use FTK Imager to create a copy of the live system’s disc and memory without installing anything in the Virtual Machine.

A key benefit of this process with FTK is to gather evidence without direct system interaction thus reducing the footprint on the compromised system. As a hosting provider this is critical because we can’t actively monitor with tools inside the OS itself. The tool Vm2dmp can also convert the DPM backup files because the .bin and .vsv file format is used to store the “System State” for protected systems. I feel that if something happens, that this type of insight can be critical with huge amounts of detail in what happened in the past for a better investigation. These backup files could reduce the data lost by an attacker using anti-forensics techniques. In most cases today the bad guys are getting really good at this process of covering their tracks so the DFIR community needs to get ahead of the problem before it occurs. Outside of virtual machines DPM can still perform “System State” backups of physical machines from Windows 7, Windows 8, Server 2008 R2 and Server 2012.

Here is some quick information on how to use vm2dmp

Create a dump file using virtual machine state files:

vm2dmp.exe -bin C:\VM\example.bin -vsv C:\VM\example.vsv -dmp C:\VM\memory.dmp

Create a dump file from virtual machine and snapshot name:

vm2dmp.exe –vm SERVER2008R2SP1-ENT-64 -dmp C:\VM\memory.dmp

vm2dmp.exe –vm vm SERVER2008R2SP1-ENT-64 –snap “vm SERVER2008R2-ENT-64 -snapshot-SP1” -dmp C:\VM\memory.dmp

Create a dump file using local symbols:

vm2dmp.exe -bin C:\VM\example.bin -vsv C:\VM\example.vsv -dmp C:\VM\memory.dmp –sym C:\symbols

For more information download the vm2dmp guide here: http://archive.msdn.microsoft.com/vm2dmp/Release/ProjectReleases.aspx?ReleaseId=3866

SCOM 2012

System Center Operations manager allows you to manage your network and give some detailed information into your environment in a central location. It supports Windows/Linux and SNMP devices alike. It allows options from an installed agent or agentless monitoring for information.

It has plenty of powerful rules that integrate into active directory and auto discovery rules to find new devices in the environment. This management server allows a centralized event log collection instead of taking up critical space on production systems. It can alert you via text or email if an event occurs based on custom rules. It even provides infrastructure information for System Administrators on descriptions, possible resolutions, and common causes.

I feel like it’s a more advanced Splunk that allows you target information with custom dashboards. I will not go into all the details because it can almost do anything and I provided some links to check out if you are interested.

For more information on SCOM check out these websites

http://blogs.technet.com/b/kevinholman/archive/2011/07/26/deploying-opsmgr-2012-a-quick-start-guide.aspx

http://www.scom2k7.com/installing-the-scom-2012-web-console-prerequisites-the-easy-way/

http://blogs.technet.com/b/kevinholman/archive/2011/07/26/deploying-opsmgr-2012-a-quick-start-guide.aspx

http://www.scom2k7.com/installing-the-scom-2012-web-console-prerequisites-the-easy-way/

http://technet.microsoft.com/library/hh278852.aspx

http://blogs.technet.com/b/momteam/

http://www.windowsnetworking.com/articles_tutorials/Introduction-System-Center-Operations-Manager-2012-Part1.html

http://technet.microsoft.com/library/hh278852.aspx

http://blogs.technet.com/b/momteam/

http://www.windowsnetworking.com/articles_tutorials/Introduction-System-Center-Operations-Manager-2012-Part1.html

 

VMM 2012

Virtual Machine Manger simply allows you to deploy fully updated virtual machines to your Infrastructure that follow a “Best Practice” standard with repeatable processes to avoid human mistakes. Even the simplest detail can be catastrophic for security protections in a complex environment. This also allows you to save time for the infrastructure guys allowing them to focus on much more important tasks.

I would love to go into more detail but I feel this post has covered enough for the day. Please feel free to leave any questions below or email me and I will respond as quickly as possible. Constructive criticism is always desired so please don’t hesitate.

I would to thank Andrew Case and Michael Hale Ligh for some proof reading and great suggestions. They did more than I expected as my first version was a little rough.

It has been quite some time since my last post so I thought I would share a small experience today from an email migration and some information I learned. I will note that this information is already readily available so nothing new here.
Well I’m sure most people have had people get cranky if they ever switched to a new outlook profile. One of the big bummers in the office was not being able to see their Auto-complete list stored in their n2k file.
I of course had to disagree well knowing that Microsoft loves to be caching information. So with a little bit of help from Google I quickly found the DIR in which Outlook caches dat files C:\Users\”Username”\AppData\Local\Microsoft\Outlook
There is also this niffy little tool that if you pay for it in a corporate environment would allow you to edit the AutoComplete list that is stored in a .dat file inside C:\Users\”Username”\AppData\Local\Microsoft\Outlook\RoamCache
NK2Edit allows you to view the file with a ton of options. The N2K viewer is fully free but in order to use the editor will cost you a few coins. (Great price for big company’s look for a quick fix to an annoying co-worker due to lost auto-complete goodness)
The default file that was picked up by NK2Edit was C:\Users\wroersma\AppData\Local\Microsoft\Outlook\RoamCache\ Stream_Autocomplete_0_9F18B280B91C584490C33E11FC630E5A.dat
Now if you download the program you will notice that these files are caching quite a bit of information to aid the user in the search of a recent contact they have been in touch with.
The viewer shows that information such as the Index number, Address type, Display Name, Exchange Email String, Drop down name, Search String, Domain, SMTP address, Record Weight, Sending format and if there is duplicates.

This is a lot of information that I found stored in plain text when I viewed it in a hex editor.

So if you can rename files you could easily use this for your own purposes. One thing I will note in my search was that these files and back log for over a year in the past with a profile that isn’t even there anymore.

Some other files of interest in the same location

I told you I would be back in no time with another dig from the archive on a file server it took me like 20 minutes to get 2 on my own network. It’s a long story but let’s say me and my virtual XTMv Watchguard firewall are not getting along. I set up a new ESXi box on my HP xw8400 workstation with the new raid card and transferred over my AD while my XTMv box got angry.

Back to forensics I actually going up the list only 1 step from last night to the 302 challenge from DC3 2011. This is the Shadow Volume Win7 Registry Analysis which was actually one of my favorites. Since the case is a registry case I used Harlan Carvey Registry Ripper v2.02 (regripperplugins_20110830.zip – version of plugins) against the provided system hive files because his program can examine registry hive files and export evidence reports into easy to read text format documents.

The challenge called for the following information provided by the Department of Defense.

I would later go on to do it with one of my favorite programs of 2011 Registry Decoder but I will save that for another post down the road because I plan to go much more into detail.

“Description: Examiners must develop and document a methodology used to determine, from the provided Windows registry files obtained from a subject’s computer (used to create the 400 – Shadow Volumes Analysis Challenge), a method for detecting items of interest in the system Registry files.  Items of interest are any items that would be non-standard or not normally found on a majority of computers; or those items that indicate activity or awareness of the user that may be of interest to the investigation.

Report the exact registry key path for each item of interest listed below with any additional entry information.  Include a detailed explanation of your processes (software or technique) used to examine and detect the information, and the reason for your selections.

Examiners must also be concerned with recording a detailed methodology of the steps and recording the tools used to accomplish the task as part of their grade.”

Methodology

Notes:

Commands are highlighted in Bold Font

Selections are highlighted with Italic Text

File Hives where placed in the directory 302-Shadow-Volume-Win7-Registry-Analysis\Files\Hives\

Spaces where edited out of directory and file names to avoid command line errors(well I’m lazy and like to avoid the pain of “ “ )

I opened (with administrative privileges) the command prompt (cmd.exe) and ran the following commands.

“Cd C:\Regripper

rip -r C:\302-Shadow-Volume-Win7-Registry-Analysis\Files\Hives\NTUSER.dat -f ntuser >> C:\302-Shadow-Volume-Win7-Registry-Analysis\Registry-Ripper-Report-NTUSER-dat.txt

rip -r C:\302-Shadow-Volume-Win7-Registry-Analysis\Files\Hives\System -f system >> C:\302-Shadow-Volume-Win7-Registry-Analysis\Registry-Ripper-Report-System.txt

rip -r C:\302-Shadow-Volume-Win7-Registry-Analysis\Files\Hives\software -f software >> C:\302-Shadow-Volume-Win7-Registry-Analysis\Registry-Ripper-Report-Software.txt

rip -r C:\302-Shadow-Volume-Win7-Registry-Analysis\Files\Hives\sam -f sam >> C:\302-Shadow-Volume-Win7-Registry-Analysis\Registry-Ripper-Report-Sam.txt

rip -r C:\302-Shadow-Volume-Win7-Registry-Analysis\Files\Hives\security -f security >> C:\302-Shadow-Volume-Win7-Registry-Analysis\Registry-Ripper-Report-Security.txt “

Alright to reduce the massive amount of information I blast into these I will provide simple short links if you want to get into the nuts and bolts of my results. Please feel free to use some of my commands in your own investigation or practice. I will warn there are far more powerful features but in this case it wasn’t really required to go after any special registry keys that the Great DFIR community hasn’t already covered for newbs like me to be able to use.

Key Files of Interest: Files of Interest 302

Registry-Ripper-Report-NTUSER-dat Registry-Ripper-Report-NTUSER-dat

Registry-Ripper-Report-Sam Registry-Ripper-Report-Sam

Registry-Ripper-Report-Security Registry-Ripper-Report-Security

Registry-Ripper-Report-Software Registry-Ripper-Report-Software

Registry-Ripper-Report-System Registry-Ripper-Report-System

Alright I tried to keep the post a little shorter for browser friendly kindness. If anyone has suggestions please feel free to contact me on twitter, G +, or email me at wyattroersma@gmail.com. I would recommend leaving comments and feedback if you have something you want to see or possibly see something I could be doing better. I Love feedback because I can’t fix what’s broken if I’m blind from the problem.

Want more Registry Forensics information? Go buy the latest version of Windows Registry Analysis Book by the RR creator himself http://www.amazon.com/Windows-Registry-Forensics-Advanced-Forensic/dp/1597495808.

If your to lazy and not convinced then go check out the start of it http://www.forensicfocus.com/downloads/forensic-analysis-windows-registry.pdf.  I mean if you enjoyed my mini low level post at all then this will be the perfect bunny hole to chase down to feed the information monster inside.

In the coming days I will be posting some more Digital Forensics Challenge madness from my long endless nights of trying to figure this crap out. So please stay tuned as I will be going into Registry Decoder (Andrew Case doesn’t know it yet but I’m pretty sure I can get some comments from him about how it all came to be from the beginning) It will be a good dive into Registry Decoder which recently got nominated for Digital Forensics Software of the year, it will be a close race with Log2Timeline. I mean the log will prolly take it with the amount of power it offers.

Alright I’m finally off to do something along the lines of learning from this web thing. Turns out it can really take the time out of your life.

A final Note:

Follow @keydet89 for the Registry Goodies

I felt like actually putting something out on a website I took down months ago. I have been busy with life crap, school and work. I am here to bring the return of my blog with a little guide or Methodology if you will of a DC3 challenge I did last year for 2011. If you are unfamiliar with the challenge what are you waiting for, a sign? well here is your Link http://dc3.mil/challenge/2012/

To get things into perspective of what the investigation indicated here is the Challenge instructions:

 

“Description: Examiners must develop and document a methodology used to examine and recover the contents of unallocated media.

The media in this instance is an image of unallocated space from a USB thumb drive.  Points will be awarded for recovering files from the image of that unallocated space and for providing the file(s) and any information on the recovered data.

Points will be awarded for the accomplishment in locating and providing the information requested, and the degree that you successfully accomplish this task.

Examiners must also be concerned with recording a detailed methodology of the steps and recording the tools used to accomplish the task as part of their grade.”

That basically cover what they instructed me to do. It also came with a raw image file “Memorex-TD-Classic.dd”

So for those looking for the quick results here they are in a CSV – Report file Files.cvs (Filelist )

If you care about how I got there well I’m getting there.

Frist a reference of all the things others created in order for a Noob like me to be able to pull something like this off.

Methodology

Notes:

Commands are highlighted in Bold Font

Selections are highlighted with Italic Text

The first thought I had was to export the file from the packed rar file they provided us to download. It also came with a provided MD5 Hash so I checked that to make sure it was the right file. (If you don’t know how to file hash I will be posting a how to later, hopefully most reading at least know this much)

I also try to use free programs for the most part however its sometimes has a place to actually use something that costs a little bit of coin.

For the Analysis my first instinct was to open the file in a hex editor and start poking around for information regarding the goodies we can use to recover it with as little effort as possible.

Hex Editor Analysis

I first started by renaming the provided file for process functionality from Memorex TD Classic to Memorex-TD-Classic which helps  avoid space errors in some programs. I also created the folder named “303” on the root of my C: drive. I then saved the new Memorex-TD-Classic in directory C:\303\Files.

I then open the file in Hexeditor Version 1.7.7.0 and examined the file and the first important evidence I came across was at offset: 0x00001803 hex value 45 58 46 41 54 which is “EXFAT” in ANSI which indicates a EXFAT file system used to be present on this image.

Understanding that exFAT file systems contain a boot sector for recovery purposes I decided to try and see if I could recover the entire image by restoring the boot sector so the computer could recognize the device. I mean why it would not be this easy to begin with.

I know of a nice little awesome program called testdisk that would easily allow me to pull this off if the recovery boot sector will work. (Note you can manually sure for the header of the boot record but I already knew it was there through my first trial solution that didn’t work)

Testdisk

A testdisk-6.12 program created by Christophe GRENIER has the ability to recover these types of file systems.

Step 1 in testdisk:

I placed this program into C:\303\ testdisk-6.13-WIP directory for organizational reasons. I opened (with administrative privileges) the command prompt (cmd.exe) and ran the following commands.

C:\Users\Triple>cd C:\303\testdisk-6.13-WIP

C:\303\testdisk-6.13-WIP>testdisk_win C:\303\Files\Memorex-TD-Classic.dd

(Screen shot  of step 1 commands)

Step 2: Select a media type

I then selected >Disk C:\303\testdisk-6.13-WIP>testdisk_win C:\303\Files\Memorex-TD-Classic.dd -515MB /492 MiB

(Screenshot Step 2 in testdisk: Select a media type)

Step 3: Please select the partition table type, press Enter when done

I selected the >[ None ] Non partitioned media because the structure is not any of the other listed formats and also a corrupted format.

(Step 3: Please select the partition table type, press Enter when done)

Step 4: Boot Sector Recovery

I then selcted the >[ boot ] Boot sector recovery option on the currently selected file Memorex-TD-Classic.dd.

(Screenshot Step 4: Boot Sector Recovery)

Step 5: Advanced File system Utility’s

I then selected >[ Advanced ] Filesystem Utils

(Screenshot Step 5: Advanced File system Utility’s )

Step 6: Copy backup superblock over superblock

Because the backup boot record is “exFAT OK” there is a boot sector backup that can be written to the main boot sector in order to restore the data in case the main boot record is damaged.

I then selected >[ Backup BS ] Copy backup superblock over superblock

 

Step 7: Copy backup exFAT boot record over main boot record, confirm? (Y/N)

Confirm the option to right the backup in order to recover the image

***Warning this will write to Evidence file

Type Y >Enter to confirm (Screenshot Confirmation screen)

Exit testdisk

FTK Imager

I then turned to AccessData product FTK Imager v3.0.1.1467 in order to mount the recovered image and export the file system in order to ensure the recovery worked.

Adding the image file C:\303\Files\Memorex-TD-Classic.dd

I then selected and right clicked the TD Classic [exFAT] to export a file hash list to C:\303\Files\Filelist.csv

I then selected the TD Classic [exFAT] to export a files… to C:\303\Files\ Memorex-TD-Classic this is evidence files exported from the image file showing that the image is intact after a unallocated file recovery.

 

You can review the Exported CSV I provided earlier but that is about it and all it took to perform a nice littler repair on some data. If anyone seen anything I missed/did wrong/ could do better please leave a comment and I will be sure to make a note of it and update the guide if need be. A few quick shout outs before I sign off into other random project I do.

I used Evernote for the documentation process and taking the screenshots. http://www.evernote.com/ if you haven’t installed it yet go ahead and give it a try, more than likely you will love it as well.

Also a quick shout out to Mark McKinnon, Andrew Case, and the others that helped me figure challenges out. The forensics community is amazing and extremely helpful when you reach out for it.

Later I’m out.