Wednesday, April 30, 2014

Weekly Lab Patching Task Sequence




We use Deep Freeze and SCCM to manage our computer labs. Previously I wrote up an overview of how we do this. With this post I will explain how we use a task sequence to deploy updates and software during a maintenance time, which allows greater control over what order applications and updates are actually installed. I find this is more desirable when under a tight time schedule like our maintenance times. Here is an example of the task sequence I use.

Overview

We run software updates from our WSUS server before we install 3rd party patches or software. You could do it the other way around, depending on what your priority's are, but since I also set every step in the task sequence to continue on error, if the Software Updates fail the TS will still make it to the 3rd party patches and additional software installs. We run three install update cycles, forcing a reboot and update scan between each to catch those updates that need an update.


Pre Patch Actions

Here is where any commands or scripts would be added that needed to be run before the start of patching.

Install Updates I, II, or III

Install Software Updates I, II, or III
A straight forward "Install Software Updates" step. Since our updates are deployed as "Available" to the update collection we want to choose All Software Updates here and not Mandatory Software Updates. We deploy them as "Available" so that the TS will install them and not be interfered with by the normal patching mechanism which would otherwise install "Required" updates during the maintenance time.




Restart Computer
A straight forward restart. The TS will pick back up after the computer restarts and the Configmgr client is ready.


 

Install Updates II

Scan for Updates II, III

 Here we run a command so the client will do a delta scan for any updates it is missing after the previous round of updates.
WMIC /namespace:\\root\ccm path sms_client CALL TriggerSchedule "{00000000-0000-0000-0000-000000000113}" /NOINTERACTIVE


Or (and I haven't experimented with this) you could run a full scan with the command:
WMIC /namespace:\\root\ccm\invagt path inventoryActionStatus where
InventoryActionID="{00000000-0000-0000-0000-000000000113}" DELETE
/NOINTERACTIVE
Wait for Scan to Finish II, III
Here we run a powershell command to wait 300 seconds between initiating the scan and running the software updates.
Powershell.exe -command start-sleep 300


 Install 3rd Party Updates

Next we have Install Application steps to patch or install 3rd party software. In this example we are installing the current versions of Flash player and a Hotfix rollup during the weekly maintenance window. I know the MS hotfix rollup is in the wrong category and is not a 3rd party update, but it gets the job done and I was on vacation that week =P

Install Additional Software

This space holder is for installing new software.

Agent Maintenance, SCEP Update

 Space holders for patches to the Configmgr client,  WU agent, or SCEP.

Post Patch Actions

 Run Hardware Inventory
We want our lab clients to report their new software installs in after each maintenance period so we can view reports to verify the installs the next morning. Its important for a classroom lab that if you need a specific version of software to perform instruction, that all computers in that lab are on the correct version. So we force a hardware inventory at the end of the Task Sequence.


WMIC /namespace:\\root\ccm\invagt path inventoryActionStatus where InventoryActionID="{00000000-0000-0000-0000-000000000001}" DELETE /NOINTERACTIVE
Wait for Scan to Finish
Our typical lab has many software titles, so I give the machines 600 seconds to finish reporting in hardware inventory.
Powershell.exe -command start-sleep 600
Restart Computer
And finally, one last reboot to clear up any pending restarts before our Deep Freeze program freezes the machines and shuts them down.

 

Problems

I have found that if you make changes to the TS or packages it references mid week before the TS runs, it may run and use an old copy of the TS instead of the current one. I believe this is because the machines were frozen when the client checks in so the machine "forgets" about the updated TS, then when it boots during the maintenance window it runs the old TS before it has time to check for updates. I am looking for a solution to this problem. I believe that running a check for changes at the beginning of the TS will fix this issue. If anyone has any idea's please share. I am wondering if forcing a "Machine Policy Retrieval & Evaluation Cycle" will cause the client to update its local cache of the TS and packages associated before running but I have not yet found any information that this action does that.

Friday, April 25, 2014

Patching Computers running Deep Freeze using SCCM in Higher Ed

Deep Freeze introduces some issues that need to be accounted for when planning a computer lab maintenance schedule with Configmgr. At the university I work for we have implemented the following solution.

Our lab maintenance time is weekly on Friday mornings from 2:15AM to 6A.M. This time was chosen because its when we could have all the computers labs out of service with the least impact on the students. Some disciplines, notably the science and engineering students have 24 hour access to their computer labs which can create difficulties in establishing a standard maintenance time for all computers across the empire. We would like to have had multiple maintenance times a week, but it wasn't possible across the whole campus and for simplicity's sake, currently we have not broken up labs into different maintenance windows.





2:15 AM - Computers turn on! We hope. We try to ensure that the computers are on during the maintenance window in the following two ways.

First, we have pushed packages that modify the BIOS on those computers that support it to turn on at the appropriate time. We have found that WOL is not always reliable in our environment, so configuring the BIOS on computers that support it is desirable. This has enabled us to wake a large number of our PC's for the maintenance time. We are working on getting to a 100% success rate, either by updating the BIOS to versions that support auto turn on, or replacing the computers with ones who's BIOS support auto turn on. We use Dell CCTK to configure the BIOS on Dell's during deployments, Dell OMCI to modify the BIOS on already deployed Dell systems, and the HP Bios Configuration Utility for HP's. These utility's are really cool, they make it so you do not have to visit the computers to configure the BIOS's.

Second, we send a Wake On Lan command from the Configmgr server to the computers in the maintenance window collection. The way we do this is by deploying a package at 2:15AM on Friday mornings. We set this deployment to ignore maintenance windows. Since at this time the computers are still "frozen" by Deep Freeze, we do not want the package to do anything, we are just gaming Configmgr to send a WOL signal by deploying the "do nothing" package. An example of this deployment is here.

2:25AM - Once the computers are on, Deep Freeze configuration is set to reboot the computers in "thawed" mode.

2:30AM - Configmgr then puts the computers into the maintenance window at 2:30AM. All the computers we are updating are a member of a collection with a maintenance window set. On the properties of the collection, under the Maintenance Window tab you can schedule this. Below is ours which lasts from 2:30AM to 5:30AM every Friday Morning.



2:35AM - a task sequence is scheduled to run every week. This task sequence updates the computers and ensures that they are getting any reboots needed before Deep Freeze "freezes" the computers at the end of the maintenance time. In order to ensure that the Task Sequence is updating the computers during its "Install Software Updates" steps and is not being interfered with by updates coming from Configmgr outside of the task sequence, our software updates are deployed as "Available" to the collection of lab computers we are updating, and not "Required". Updates will only be installed to a collection if they are deployed to that collection, either available or required. If the collection does not have any updates deployed to it, then the "Install Software Updates" step of the Task Sequence would not install any updates. If we made the updates "Required" some of them might try to install during the 5 minute window between the start of the Maintenance Window at 2:30AM and the start of the Task Sequence at 2:35AM.

The Task Sequence looks like this, I will go into more details of each step in another post.


5:30AM - the Configmgr Maintenance Window ends, we may push this back to 5:45 to give another 15 minutes of maintenance time.
6:00AM - Deep Freeze shutdowns the computer, and the next time it boots it will be frozen.

Pushing update packages vs. the control of a task sequence

We use to just push update packages at the lab collection to update the computers instead of updating via a task sequence. We found that some updates would need a reboot to finish installing but not force a reboot, and this could cause problems when using Deep Freeze. When Deep Freeze "froze" the machine at the end of the maintenance time and the computer would boot up in the morning, the update would continue updating, and then reboot while updating. Since the machine was frozen, this would put it into a continuous loop of rebooting and updating. Not an ideal situation for a classroom or lab. I had originally attempted to fix this by running a reoccurring script that would force a reboot after the Configmgr maintenance time, but before Deep Freeze "froze" the computer. This ended up not working and the errors in the logs seemed to suggest the computer was waiting for a reboot before it would run any more deployments, including the deployment of the reboot script. The task sequence appears to handle reboots much better and gives greater control over the order that software and updates get installed and when commands are run, like the hardware inventory command that runs at the end of the task sequence.

Using a SCCM 2012 package to WOL computers for maintenance

While being able to send WOL packets to computers before software deployments, Configmgr 2012 lacks a built in scheduled task for waking computers at a specific time. A work around I am using is a "do nothing" package scheduled to run once a week. The deployment for this package is set to run outside maintenance windows, to ensure the computers are wakened and ready by the time both Deep Freeze and later Configmgr are ready to put them into thawed mode and the maintenance window. See this post for how the maintenance window timeline works at our university.

The "do nothing" program I use is DFC.exe which is a deep freeze console program, the command line I use is "DFC.exe get /ISFROZEN" which returns a value when run from command line. I don't really care about the value it returns as I am only gaming Configmgr to wake the computer before running the command. You could just as easily have it run a .cmd file that does nothing and exits. The package properties look like:

Now on the deployment for this package, its important to check the Send wake-up packets checkbox on the Deployment Settings tab.
 
 Here I have it scheduled to run every week on Friday at 2:15AM. I have set the Rerun behavior to Always rerun program.
 
 Lastly, I want this package to run outside (ignoring) the Configmgr maintenance window, so I have checked this box: