Data Processing

I have been watching many videos on early-era computing and they have led me to give a lot of thought to the fundamental concepts in computing we now take for granted.

For example, sometimes we say “data processing” when we speak about our field, but the origin of that term is often long forgotten. The first computers were not used for watching videos or browsing websites; Instead, they were primarily used for solving large mathematical functions that would have previously taken teams of humans weeks, if not months, to calculate. These computing systems were single-task machines and had about the same general functionality as a modern Central Processing Unit (CPU). “Memory,” as we presently think of it, was in short supply and data was stored on punch cards. One of the first major uses of a computing system was to help tabulate the United States Census. It sped the entire process up tremendously.

Put simply, computers at that time were used to process data. That is all they did. That is all they were capable of doing. It was not for many years and decades that we were we able to build on this core concept to arrive at the computing systems we have today. All of which is built on processing data, manipulating it, and storing it again. 

How to Speed Up a Slow External Drive

Problem

A while back a user reached out to me describing a problem of slow access to an external, bus-powered hard drive they had purchased only half a year ago. They said it was a USB 3.0 hard drive and they had also made sure to plug the drive into a USB 3.0 compatible port on their recently purchased laptop. The user also mentioned that the anti-virus solution they were using had unusually long scan times, sometimes running for over 10 hours.

They also described an issue of not being able to properly eject this same external hard drive after using it at the end of the day, but that was a separate issue that will also be covered.

Troubleshooting Methodology

After gaining remote access to this system, so I could see what they were seeing, I checked the configuration of the laptop. They were right, the laptop was powerful with a nice quad-core Intel process, 8 GB of RAM, and a SSD hard drive. But, all of this had little to do with why this external hard drive enclosure, which was a spinning disk, was performing poorly. I loaded the contents of the drive in Windows File Explorer and found multiple folders at the root of the drive. The user began navigating into the the folders and subfolders to find some files they were having issues working with. We navigated down five and six levels deep, and at each level I saw many other folders within each directory. I thought I had spotted the first issue, an index that was far too large to be accessed quickly.

Navigating back out to the root of the external drive, I checked the properties of the folder in which we had just explored and found that while it was not large in size, it had tens of thousands of files and folders within the folder. We checked a few more folders together at the root of the drive and they were the same way, tens of thousands of files and folders within each one. I explained that the drive was formatted as NTFS and that this type of file system kept a Master File Table which was basically an index of every folder and file on the disk. As this Master File Table became larger and larger as times went on, it can also became fragmented. This fragmentation could drastically slowdown the load times of folders and files within folders, because the actuator that controlled the read/write heads would have to constantly bounce around the disk to enumerate the files and folders with all their attributes within a specified directory.

Resolution

We set about resolving this issue by lowering the overall number of files and folders on the disk. We used an application called 7-Zip to compress one of the folders at the root of the external drive and then deleted the original folder from the drive. This lowered the number of entires in the Master File Table, increasing performance almost immediately. Since the user had mentioned that they were seeing incredibly long scan times with their anti-virus solution, I also recommended we password protect the zipped files, which would keep their anti-virus solution from being able to scan the contents of the file.

Over the course of a few days the user managed to compress and password protect many unused folders at the root of the external drive. They reported back much faster performance of the external drive and the anti-virus scans were no longer taking unacceptable periods of time to complete.

Bonus: Cannot Safely Eject External Drive

We had one last issue to tackle. The user was still having an issue ejecting the disk safely after each use. We plugged in the external drive and were immediately able to safely eject the external drive. We systematically opened files on the external drive with each application they used to perform their work, saved the file, and then tried to eject the drive. Everything went smoothly until the user opened an AutoCAD application file, saved the file, and exited the program. The drive would no longer safely eject. We closed a “helper” program for AutoCAD we found in Task Manager and the drive safely ejected. I showed the user this workaround method and also mentioned that a reboot would allow them to safely eject the drive, too.

 

New Upcoming Feature in Windows: Controlled Folder Access and More…

This fall, just a few short months from the time of this writing, Microsoft will be releasing a minor update to follow the most recent Windows 10 Creators Update from earlier this year. It will include some new features, including a few that revolve around their built-in Windows Defender suite. With these changes to Windows Defender, Microsoft hopes to make their latest operating system more resistant to ransomware attacks which have become prolific over the last several years.

One of the features coming with the update is called Controlled Folder Access. Microsoft touts the feature as a direct response to ransomware. It will work via a whitelist approach, with Windows Defender only granting certain applications the privilege to access the data of a protected user account; otherwise, the application is not allowed to read, write, or modify any data a user might own such as documents, pictures, or videos.

The default folder list includes Documents, Pictures, Movies, and Desktop and are hard-coded into the feature with no option for removal, but additional folders can be added manually through the Windows Defender Security Center. There will also be an option to add custom software to the whitelist, but Microsoft states that most software should already be pre-whitelisted. If an application is not whitelisted and attempts to alter data within a protected folder it will be automatically blacklisted and the user will be notified. Although this feature has many benefits, Microsoft will have the feature disabled by default. It can also be enabled in the Windows Defender Security Center under Virus & threat protection settings, as seen below.

How to enable Controlled Folder Access in Windows Defender Security Center Diagram
Controlled Folder Access settings window, courtesy of Microsoft Blog.

Other features coming with the Fall Creators Update include a Cloud Clipboard which will allow copy and pasting between multiple Windows 10 devices; a Timeline feature, which will be similar to the app switcher found on many mobile phone operating systems; Pick Up Where You Left Off, which will be an application synchronization service that developers can use much like the Cloud Clipboard; and OneDrive Files On-Demand, which will allow access to files, even if they are only stored in the cloud and not locally.

Windows 10 is also getting a design language refresh. Microsoft is moving away from the Metro UI to offer a more consistent, depth-enabled interface with lighting and motion effects. It is being likened to Google’s own Material Design. Overall, a welcome change, but one that may be more resource demanding.

Will you be upgrading? What feature do you look forward to most? Leave a comment below!

How to Update to Windows 10 Creators Edition

Microsoft recently released their next major evolutionary update to Windows 10 called Creators Update. It boasts a lot of new features and innovations. New additions in the update include Paint 3D, Remix 3D, and greater support for augmented reality with Windows Mixed Reality. Some new features include better Cortana integration with more apps, the ability to easily broadcast games live with Microsoft’s Beam service, and blue light reduction (like Night Shift in iOS). Edge even got a feature update with tab grouping.

There is also a larger feature that has gamers excited called Game Mode. There has been some early testing done and it was found that enabling this mode with slower, cheaper processors did see a performance increase while gaming.

As far as new security features, Microsoft is investing heavily in Windows Defender. They have given Windows Defender its own home with a new applet called Windows Defender Security Center. It gives much more information and allows for overall easier management of security features. Windows Defender Security Center includes other security measures for management, such as Windows Firewall and browser security settings. Additionally, Windows Hello now allows for proximity auto-locking when paired with a bluetooth device. Microsoft also added a privacy dashboard which will allow one to manage activity data across multiple services and more simply change privacy settings.

All of these features and updates are exciting for anyone interested in technology. But, before one dives straight into the update, even if offered automatically, there are some things one might consider doing first. Below is an outlined process by which one might go about performing the Windows 10 Creators Update.

1. Preflight Check

There are certain things that should be completed as a precaution before performing this upgrade to Windows 10 Creators Update. First thing, always make a backup. Disk2vhd is exceptionally good for this and is available for free from Microsoft, included as part of the Sysinternals Suite. Disk2vhd could not be more simple to use. Once you download, unzip the file, and run it, follow these directions:

  1. Be sure to check the Use Vhdx and Use Volume Shadow Copy options.
  2. Choose a path where to save the image file and give it a name.
    • The author strongly recommends a USB 3.0 external drive with adequate capacity on which to store the image, when considering a path.
  3. Check the box of the appropriate volumes which should be included, excluding any external volumes.
  4. Click the Create button.
Disk2vhd
Disk2vhd utility from Microsoft.

Depending on how much data is on the volume(s) selected and the transfer speed of the drive used, the time required to complete the backup can range from 30 minutes to several hours. Browse into the image once it is created to make sure it is not corrupt and your files will be accessible, in case the update procedure fails. This can be done by right-clicking the drive and selecting Mount, then opening Disk Management and right-clicking the mounted Disk and selecting Online. When done, in Disk Management right-click the mounted Disk and select Unmount. An additional prompt will appear, just click OK.

Just for safe measure, be sure to unplug the external hard drive after safely ejecting it.

2. Start the Update

If the Creators Update does not appear automatically, it can be started by browsing to the following Microsoft Creators Update page and clicking the Update now button.

This will download a small utility which starts the update. Launching the utility brings up the Windows 10 Update Assistant. Click the Update now button. This will perform a check to see if your system passes the requirements for the update. The utility will then begin downloading and installing the update automatically. Depending on the speed of your connection to the remote server (the Internet), this download can take several minutes to several hours. Feel free to work in the meantime and minimize the window.

Windows 10 Upgrade Assistant
Windows 10 Upgrade Assistant window.

Once the update process is completed, the Windows 10 Update Assistant will prompt for a restart of the computer. It also kicks off a 30 minute countdown timer, so be sure to be close at hand when this part of the upgrade process approaches. An unexpected, automated reboot could cause loss of data.

3. The Update Process

Below is a video of the update process. There are several reboots and it took almost an hour to complete on a 2014 iMac with an Intel Core i5 processor and 1 TB spinning disk drive.

4. Post Update

After the update is finished, you will be guided through some of the new settings, displayed below.

First Screen
First screen after last reboot of update.
Privacy Setting Page
Privacy settings configuration screen.
Cortona Setup Screen
Cortona setup screen.

5. It is Complete

After everything is complete, if there are no issues, you will be presented with the following screen.

Windows 10 Creators Update Thank You
Windows 10 Update Assistant Thank You screen.

Noticeable Changes

I immediately noticed that the iMac on which this upgrade was performed seemed to run much quicker than it previously had. It is hard to tell if this is due to the optimization of Windows 10 in this update or because the update was like refreshing the system. Applications seem to launch faster and boots seem quicker, as well.

What results have you noticed after the update?

Post a comment below with anything you experienced or what you think about the Creators Update.

10 Steps to Securing The Endpoint

 

Endpoint security is often not given a lot of thought or planning within an organization, but it can be incredibly important. For most security-minded people, endpoint security is usually associated with technical controls. These controls are often not perfect, but are a good first line of defense to help build a more secure network. If you, as a Network or Systems Administrator can put these controls in place, it will make your life much easier over time, because a lot of breaches begin with downloaded malware that is installed by the end user, with or without their consent. Here are 10 technical controls you can implement on your network to help mitigate that risk.


Standard User Account

This control can be implemented in almost every operating system or light-weight directory environment. It returns results immediately, as well. The gross majority of malware requires administrative rights to install, even those that do not require user consent. By using a standard account on all endpoints, as an administrator, you greatly limit the attack surface that account would have otherwise provided running with administrative rights.

Account Auto-Lock Policy

It is pretty common, most users do not lock their workstation or laptop when they walk away. But, by having a policy of locking endpoints when they are not in use, you increase user authenticity and bolster physical security. In a Microsoft Windows environment or network, this policy can be easily enforced through Group Policies to all machines on a domain. In a Macintosh environment and network this can also be achieved with Profile Manager and a Macintosh Server. (Microsoft’s Systems Center Configuration Manager 2012 R2 can also manage Apple Macintosh systems, as well as Microsoft Windows systems.)

Operating System Updates

Operating system updates should be top priority in any environment. Almost every major operating system, as of this writing, has a method for automating updates for at least critical, important, and recommended updates (naming scheme is often platform dependent). These updates can also be automated with different centralized management platforms, paid and free. Usually, the easiest way to centrally manage updates across a network is to use the vendor’s configuration utility (mentioned under ‘Account Auto-Lock Policy’). A popular, free solution is Windows Server Update Services (WSUS) by Microsoft. Apple’s Mac OS X Server can also act as an update proxy server.

Third-Party Software Updates

Patching third-party software is usually a little harder to automate, but is just as important as operating system updates. Third-party patching can be harder to perform due to different patching schedules, multiple vendors, and support, or the lack thereof, in patching clients. One of the author’s favorites at the time of this writing is Ninite. Ninite Pro can even be implemented across multiple domain-joined machines in an Active Directory environment. But, there are many other options like Shavlik and GFI’s LanGuard.

Host-Based Anti-Malware

Anti-malware software has gotten a bad rap over the last few years, because most users consider it inadequate. Truthfully, it is by itself; that is why it is only 1/10th of all of the controls in this the recommendation list. Anti-malware software should be used on an endpoint as the first line of security, and considering it comes with most major operating systems, it should be enabled no questions asked. It should acts as the watchdog on the system and any alert that it finds should cause an escalation of the effected system for further analysis. The author recommends Cylance, at the time of this writing, for Windows environments.

Host-Based Firewall

Once again, at the time of this writing, almost every operating system comes with a built-in firewall, much like automatic updates and anti-malware software. It should just be enabled. Again, it can be centrally managed in most environments and third-party vendors have offerings that can be centrally managed, as well. Use it. It is better than nothing.

Install Secure Browser

This should be everyone’s favorite topic, because of all the great, free choices. Personally, the author recommends Google’s Chrome browser at the time of this writing. Google Chrome uses pinned certificates (to combat inauthentic certificates), each tab is a separate process, there is great extension support, and the built-in synchronization settings make everyday tasks almost operating system-agnostic. Mozilla’s Firefox is another great choice, but is relatively less secure by most counts. Internet Explorer honestly should not even be a choice in 2016. Microsoft’s Edge browser is still a little too new and has no extension support. Apple’s Safari is adequate, fairly secure, but lacks the speed and extension support of Chrome.

Full Hard Drive Encryption

Hard Drive encryption is available on most operating systems, if the correct edition is used. Microsoft offers BitLocker and Apple offers FileVault. A good third-party, open-source alternative with Trust-No-One (TNO) security is VeraCrypt. It is best used in situations where physical theft is a risk in the organization. It is highly recommended for portal computers such as tablets and laptops, but it can also prove useful for workstations. Encrypting external hard drives and thumb drives is also highly recommended.

Remote Logging

After any breach or security event, logs are always one of the first ways to track down exactly what happened. The only issue is, if logs are located on the same system that was compromised, the logs can no longer be trusted. The best option is to move the logs, or copy them, to a remote system. Usually, there will be a server (preferably two) that collects the logs from all of the other systems on the network and store them securely. Loggly has an open-source solution, for those looking to get started.

External Backups

Having a good backup should be an integral part of every network. All endpoint data should be located centrally and then protected by a standalone backup appliance. If endpoint user data must be decentralized, even a Windows Backup on an external drive is better than nothing. Getting those backups to an off-site location should also be a top priority. If endpoints are joined to a domain and important files are replicated to a file server, then Datto makes a good selection of standalone backup appliances that replicate data to highly-secure, off-site redundant bi-coastal datacenters.


As an administrator of any type, if you can implement half of these recommendations, you will be doing better than most. But, even if you implement all 10 steps, this is by no means a guarantee that nothing bad will happen. These are 10 strong recommendations to help mitigate risk. If you put these controls in place it will make your life much easier and your network much safer.