Copenhagens Setech

Lenovo fingerprint software problems, a ATA locked HDD i can not access

2009-04-30T00:03:00.005+02:00

Lenovo laptop user

As a daily user of several ThinkPad Lenovo and the older IBM laptop computers (at work and at home private) I am sorry about the experience I recently had with Lenovo!

I am sorry for 2 reasons:

I expect there to be some problems with the Lenovo fingerprint software.. not just any problems, but some that has now made my harddrive data inaccessible.. ie. I have lost data that i do not have backed up! Fortunately that is not much data, because I upload frequently to flickr and burn backup to dvd!..
When trying to get help with the data problem, and discuss what happend in order to avoid and explain, on the Lenovo forums, I got 2 good replies, then without notice the thread was deleted. Now we might never know if there are indeed flaws in the Lenovo software, if there is a solution that can recover or bring back my data or my old XP installation..or it was simply just me that used the fingerprint software wrong!
It was very helpful replies that explain some of the problems and possible reasons, so deleting the texts is very annoying for me! And the discussion was ongoing, I still have questions that was left unanswered. All the answers could have been helpful for others in the future..

Original Lenovo forum posting

Well for my future reference, I will save the information here, so i have it, and maybe someone else will find it useful:

I posted the following question on the Lenovo forum:

Hi

I am having trouble mounting and accessing data on my old lenovo r60 thinkpad hdd which had xp before.

what happened:
1) i decided to disable the fingerprint software because i had been in the garden and my fingers was scratched so it kept failing, but obviously i got in after many retries
2) i chose restart computer and it booted up and i had to logon with password. all looked good and i was happy

3) next day i started computer, and now it asks for a password i dont have. the password it asks for is illustraded with what looks like a Database icon and has a number 1 shown. Problem is, I do not have that password and i dont even remember setting it

4) So i took the old disk out of my R60, putted in a new disk, and installed windows xp on this new disk
5) Inside Windows XP, i try to mount the old disk using usb, but xp only see the disk, it sees no partitions
6) I used some recovery tools, because i thought all of a sudden the disk had gone bad,.. which would be really bad luck, it has been working fine for a long time. But anyway, when i scan for partitions, software finds no partitions, and the errormessage is a lot of "read error" for cylinders.

It is almost as if the disk has been encrypted somehow, but that is not something i know or have done myself.

Is this some kind of security setting from Lenovo software that i can undo directly on the disk?

Or can i get around the password prompt and just boot back into my old Windows XP installation?

Or is the disk just plain defect all of a sudden...

First reply
The first reply i got, pointed me in the direction of "It sounds like you have a ATA harddisk password lock on the disk". Of course now that reply is deleted, but i think the most important was something like:

It sounds like you have a ATA harddisk password lock set

The fingerprint software has supplied the ATA HDD unlock password, and when you made the hot boot in 2) you didnt notice any problem because the drive only needs unlock once it has been powered on.

After the cold restart you need to enter the ATA HDD password because the fingerprint software no longer does it for you. Unfortunately you will be out of luck trying to enable the fingerprint software again as you can not boot the drive.

Alright this made very much sense to me, it sounds exactly like i have had some ATA password set. The problem is i dont remember setting it in any Lenovo software and I dont remember what password was used.

My Reply

So i replied to the thread with some clarifications, questions and suggestions for improvements of the Lenovo fingerprint software.. that reply is also deleted from the forum ofcourse, but it was something like:

When the old disk is attached with external USB cable, XP device manager shows the disk, but inside Disk Management there is no disk to work with.

I used an open source free partition recovery tool (which i used in the past to restore a mbr on a drive that would not boot) TestDisk [http://www.cgsecurity.org/wiki/TestDisk], which does show the disk, but analyzing the disk gives just a lot of read errors.

I also tried scanning the attached disk with a free trial of File Scavenger, but that also gave no results. So i am afraid it is a ATA lock password problem.

1)
It bothers me that that I disabled the fingerprint software, only because it took me many retries after I had been to the garden and got some scratches. I couldnt figure out how to change the only 3 retries of striking the finger, is it possible to change that? It bothers me because i would give a lot to be able to turn fingerprint security back on now, but I cant because the drive is locked!! It would make much sense to me if Lenovo had a tool that could turn on fingerprint security again.. WITHOUT going into windows of course!?

2)
It also bothers me that the fingerprint software allows me to turn off fingerprint security without giving me some kind of major warning: WHEN YOU TURN OFF FINGERPRINT SECURITY, YOU NEED THE ATA HDD PASSWORD IN ORDER TO BE ABLE TO START WINDOWS AGAIN.. ARE YOU SURE YOU HAVE THIS PASSWORD OR DO YOU WANT TO SET A NEW OR REMOVE THAT PASSWORD? ... how simple would this be.. and how nice ...could have saved me data loss... I mean after all the Lenovo fingerprint software must know, or be able to detect, that there is a HDD lock on the disk because the software has handled the single-sign-on for drive unlock until now!!! sigh

3)
Finally it bothers me that the Fingerprint software does not have a clear tab for "setting/changing the ATA lock password".. If there had been such a place i would have seen it, and possibly i would have changed the password, or even turned it off!!

I feel these texts inside the thread are very important for me and other Lenovo endusers... too bad the thread has been deleted.. very sad judgement from the Lenovo forum administrators or moderators or whoever deleted it:

Disk with Lenovo fingerprint protection cannot be mounted on another Windows XP

I hoped to get help from Lenovo about this, but no.. and not only do they not help, they even worked against me by deleting my post for help :-( what a pity ..but i will keep using the Lenovo laptops as I have for many years ... this is just a bit sad, and of course i will never use the Lenovo fingerprint software ever again..

So.... anyway, i have some data on that drive which are of very big importance to me, so i will continue for some days to find a solution, possibly paying for a unlock of the drive [http://www.hdd-tools.com/products/rrs/] which seems to be the only way:

With Repair Station you can:
remove an unknown ATA-password; both security levels are supported: High and Maximum
diagnose and recover HDD firmware area
ATA Passwords
...
Repair Station has the ability to access the Firmware Area and reset the password, thus making your hard drive unlocked. Unlocking process is done automatically and takes just a few minutes.
Since Repair Station does not alter partitions or file systems, it is absolutely safe to your data.

There is a free diagnose trial of the software, but it requires that the disk is mounted on motherboard and not with USB cable. When I get a chance, I will run the diagnose..

Java Runtime Environemtn 1.6.0_10 can not be uninstalled

2009-03-17T14:27:00.003+01:00

I just had a bad experience with a program (JRE 1.6.0 update 10) that was screwed up in my Windows XP installation, possibly during a half or semi good installation! The problem is now that the software does not uninstall from Add-remove programs or from Java commandline. So I can not uninstall the program and I can not install a newer version!

Making things worse is that since this is on a computer where software updates are in controlled environment, the IT support function said that solution is to reinstall the entire computer!

Instead of doing the reinstall of the computer rigth away, I used a Microsoft guide that promises to solve just the issue i am experiencing:
http://support.microsoft.com/default.aspx?scid=kb;en-us;290301 When you are working on your computer and installing a new program, the installation suddenly fails. Now you are left with a partly installed program.
You try to install the program again, but you are unsuccessful. Or, maybe you have problems trying to remove an old program because the installation files are corrupted.

Do not worry. Windows Installer CleanUp Utility might be able to help. You can use the utility to remove installation information for programs that were installed by using Windows Installer.
Be aware that Windows Installer CleanUp Utility will not remove the actual program from your computer. However, it will remove the installation files so that you can start the installation, upgrade, or uninstall over.
And guess what, it really worked and solved the problem!! After using the Windows Installer CleanUp Utility I could reinstall and then remove, and get the newer version of Java Runtime.

Ultraedit files does not show in tabs even when set for it

2008-11-10T20:29:00.000+01:00

Every now and then my File Tabs in UltraEdit disappears. I think though only after a hard reboot crash of my computer. But as it happens often, I save the very simple solution here, easy for me to find later!

Beware, it does not work to simply enable the "View -> Views/Lists -> Open File Tabs" from the menu, that is already enabled, and no file tabs are shown!

So it turns out that all you have to do is delete this section from uedit32.ini:

[ToolBarState1-v110-Bar17]
BarID=59423
Horz=1
Floating=1
XPos=4
YPos=992
Bars=3
Bar#0=0
Bar#1=143
Bar#2=0

You can remove more [ToolBarState1 sections if you want, you will have to fix your toolbars afterwards though.

Uedit32.ini location can be modified, which is a nice feature, enabling you to have your own private .ini file. Simply using the documents and settings location ("C:\Documents and Settings\USERNAME\Application Data\IDMComp\UltraEdit\uedit32.INI") is better than Windows system dir.

Commands from cmd does not set errorlevel as you might expect!

2008-06-13T08:59:00.005+02:00

It might not be a surprise to you, but Windows commands inside cmd.exe does not change errorlevel as you might expect.

For example running a echo something > c:\somefile.txt, which will succeed actually creating the file, but not change errorlevel to 0. You can test it like this:

md 2>nul
echo %errorlevel%
1
echo this.works > c:\test.txt
echo %errorlevel%
1
type c:\test.txt
this.works

This echo can not really be solved by using cmd /c echo because that will just always succeed, for example:

md 2>nul
echo %errorlevel%
1
cmd /c echo this.works > c:\test.txt
echo %errorlevel%
0
type c:\test.txt
this.works
cmd /c echo this.fails > drivedoesnotexist:\test.txt
The filename, directory name, or volume label syntax is incorrect.
echo %errorlevel%
0

And now, testing if copy command file1 + file2 into file3 gives errorlevel 1 if one of the source files does not exists. Errorlevel 1 is what you might expect, but it is not the case here:

echo 1 > 1.txt
rm 2.txt
echo 3 > 3.txt
ls -la 2.txt
ls: File or directory "2.txt" is not found
copy /b 1.txt + 2.txt + 3.txt 123.txt
1.txt
3.txt
1 file(s) copied.
echo %errorlevel%0

This is not as I expected, I will want to find a way to get around this.

There are probably same problem with other cmd commands, I didnt try others.

Maybe I am doing something the wrong way, in my environment and installation ... need to investigate :-)

I have not been able to find anything in the cmd command line reference, and it does not seem to solvable if everything is put into a batch script, instead of running commands one by one. I did hope that, because of text on information about setlocal ENABLEEXTENSIONS which can be set in a script, but has no effect on the command prompt:

cmd does not set the ERRORLEVEL variable when command extensions are
disabled

But unfortunately it did not work, here is the run.cmd script i ran:

setlocal ENABLEEXTENSIONS
echo 1 > c:\1.txt
rm c:\2.txt
echo 3 > c:\3.txt
copy /b c:\1.txt + c:\2.txt + c:\3.txt c:\123.txt
echo %errorlevel%
endlocal

The above echo'd 0 and the errorlevel after the script is 0. So not a solution!

I still keep investigating :-)

Oh yeah - in case you ever wondered, you should never manually set the errorlevel to 0 or 1 or whatever you need. Instead you should always use a command for that. I am using "ver" to get errorlevel 0 and "md;2>nul" to get errorlevel set at 1, which I found on one of my favorite batch example webpages.

Dig into the Active Directory information store

2008-05-20T09:11:00.000+02:00

I never needed to automated users and groups creation/deletion/changes in Active Directory on Windows. I have however needed to query lists of users and groups, membership and such. That was solved by some vbscripts.

A few days ago I needed to show the list of computers in an Organizational Unit (OU), so I searched for some ways to get that. And I bumped into the Microsoft dstools (dsget/dsquery/....) and they are just perfect for automating Directory service stuff.

The commands works from a Windows 2003 server, but not from XP.

Also, I need a intro for directory services, because I havnt used it much, and a then moved to a good simple dsquery tutorial.

Then I could make some quick oneliners, starting with a very comprehensive query that is highly educative of how the Directory of Level2OU is made:
dsquery * OU=Level2OU,OU=Level1OU,DC=domain,DC=domainext -limit 0 -attr *

Listing the members of a Windows group:

dsget group "CN=somegrp,OU=level2,OU=level1,DC=domain,DC=domainext" -members -expand

Show the computers of an OU:

dsquery computer OU=Level3OU,OU=Level2OU,OU=Level1OU,DC=domain,DC=domainext -limit 0

To automated the query, I have used psexec to run it on a remote server, with a user that has access to do queries:

psexec \\srv -u dom\usr -p pwd -e cmd /C "dsquery ou domainroot"

Other than that I just found the dstools to be very powerful and some googling shows many good examples of what people have done with it! Very impressive!

The article also mentions some need-to-have directory service binaries from joeware.net/freetools but I havnt tried them. They look good though, like lots of work has been wrappen into those exes:

AdFind [switches] [-b basedn] [-f filter] [attr list]

basedn RFC 2253 DN to base search from.
filter RFC 2254 LDAP filter.
attr list List of specific attributes to return, if nothing specified returns 'default' attributes, aka * set.
...

Dependency Walker commandline example

2008-05-19T09:33:00.006+02:00

Dependency Walker (depends.exe) can be downloaded, or is in Windows 2003 resource kit.

I wanted to script the dependency check for some .dll files, so i ran toward remote server:
psexec \\someserver /u someuser -e cmd /c "environment.cmd&depends.exe /c /pb /oc "d:\depends.temp" "some.dll""

Now parse the output file, first column is status, look for "E,":
findstr /bic:"E," \\someserver\d$\depends.temp
if errorlevel 1 echo all OK

I have not made a way to avoid enter password, but if I need I recall there are some runas and similar alternatives.

Hello World and 99 Bottles of Beer collections

2008-05-08T10:14:00.002+02:00

Browsing around for some good C++ sample code I stumbled upon the The Hello World Collection. While that is good, the 99 Bottles of Beer song collection is stunning, informative and fun:

... the collection of the Song 99 Bottles of Beer programmed in different
programming languages. Actually the song is represented in 1200 different
programming languages and variations. For more detailed information refer to
historic information.

I went to look for the Perl example for the song, and was completely baffled!! Everything written inside regular expressions, using a perl module which creates shapes out of perl code! Andrew Savige has some serious coding skills!!

Checking the reg exp, it is a bit (but not much really) readable:

perl -MO=Deparse 99-bottles.pl
'' =~ /(?{eval"\$==pop99;--\$=;sub\n_\{(\$;=(\$=No).\" bottle\".\"s\"x!!--\$=.\" of beer\").\" on the wall\"\}print+
_,\", \$;!\nTake one down, pass it around,\n\",_,\"!\n\n\"while++\$="})/;
$: = 'P';
$~ = 'h';
$^ = 'r';
$/ = '`';
99-bottles.pl syntax OK

More good Windows command line tools

2008-04-30T11:53:00.004+02:00

Once again I am surprised to see more useful commandline tools, already in Windows.

When I was playing around with Powershell I stumbled upon Windows Command Reference, a .chm file with reference for a lot of command line utils in Windows:

The Windows command-line tools are used to perform various tasks related to
Windows Vista, Windows Server 2003, and Windows Server 2008.You can use the
command reference to familiarize yourself with new and enhanced command-line
tools, to learn about the command shell, and to automate command-line tasks by
using batch files or scripting tools.

Many of the tools in the reference are also in Windows XP and 2003, but the resource kit tools are not listed, for example jt.exe or tail.exe is not in the list.

With all these nice utils, and more to come probably, I am thinking a lot of old selfmade scripts can be replaced or simplified. I prefer to use windows builtin tools if possible, most often wrapped somehow.

Of course there will always come new needs, ideas for improvements, so script wrapping, script/batch control is just as much wanted as before!

Two of the utils I can use immediately, its tasklist.exe and taskkill.exe, which can query and kill processes depending on lots of different restrictions.

One of the good filter options is username, memusage and session number. Unfortunately only on one server at a time:

TASKLIST [/S system [/U username [/P [password]]]]
[/M [module] /SVC /V] [/FI filter] [/FO format] [/NH]

Description:
This command line tool displays a list of application(s) and
associated task(s)/process(es) currently running on either a local or
remote system.

Parameter List:
/S system Specifies the remote system to connect to.

/U [domain\]user Specifies the user context under which
the command should execute.

/P [password] Specifies the password for the given
user context. Prompts for input if omitted.

/M [module] Lists all tasks that have DLL modules loaded
in them that match the given pattern name.
If the module name is not specified,
displays all modules loaded by each task.

/SVC Displays services in each process.

/V Specifies that the verbose information
is to be displayed.

/FI filter Displays a set of tasks that match a
given criteria specified by the filter.

/FO format Specifies the output format.
Valid values: "TABLE", "LIST", "CSV".

/NH Specifies that the "Column Header" should
not be displayed in the output.
Valid only for "TABLE" and "CSV" formats.

/? Displays this help/usage.

Filters:
Filter Name Valid Operators Valid Value(s)
----------- --------------- --------------
STATUS eq, ne RUNNING NOT RESPONDING
IMAGENAME eq, ne Image name
PID eq, ne, gt, lt, ge, le PID value
SESSION eq, ne, gt, lt, ge, le Session number
SESSIONNAME eq, ne Session name
CPUTIME eq, ne, gt, lt, ge, le CPU time in the format
of hh:mm:ss.
hh - hours,
mm - minutes, ss - seconds
MEMUSAGE eq, ne, gt, lt, ge, le Memory usage in KB
USERNAME eq, ne User name in [domain\]user
format
SERVICES eq, ne Service name
WINDOWTITLE eq, ne Window title
MODULES eq, ne DLL name

TclockEx is still a better Windows Clock

2008-04-27T06:49:00.005+02:00

Similar to the awesome util Printkey 2000, there is another Windows util that is still my favorite for the task.. even that it is very very old: TclockEx.. a much better Windows task bar/system tray clock!

It serves a few important purposes for me:

Display date and time in a better way
Customizable format of "date and time copy to clipboard" when i doubleclick the systray. I use this format to get a quick datestring for usage in reports and documentation: yyyyMMdd-HHmm (eg. 20080427-0719)
Display a simple calender with week numbers, shown by single click and week start can be modified to monday.

The original site in the About page http://users.iafrica.com/d/da/dalen does not work, instead I used http://www.rcis.co.za/dale/tclockex/.

For the paranoid people, here is my md5 sum for the safe exe:

1238b1c59fd4987d538144aa915e85c2 *tclockex-1.4.2.exe
1238b1c59fd4987d538144aa915e85c2 *tclockex.exe

Read more about alternatives to tclock here:

http://www.techsupportalert.com/best-free-tray-clock-replacement.htm

Windows forfiles.exe, similar to unix find

2008-04-21T14:27:00.001+02:00

I have unix tools available on my Windows boxes, so I have have a tendency to use those, eg. using unix find to delete files older than x days.

Now a while ago I saw that there is a forfiles.exe in NT ressource kit, which can do similar job, only it does not work on UNC paths:

forfiles /P file://machine/share /M thesefiles*

ERROR: UNC paths (file://machine/share) are not supported.

You can get inspired by some cleanup examples:

forfiles /p C:\documentService\bin /s /m trace*.* /d -120 /c "cmd /c del @path"

forfiles /p D:\DocumentStore\imagingShare /s /m *.tif* /d -120 /c "cmd /c del @path"

And as always ss64.com has forfiles examples.

Another example for deleting files:

FORFILES /p C:\filename /s /m *.* /d -3 /c “CMD /C del /Q @FILE

File size, file count, file age - batch util

2008-04-17T09:32:00.004+02:00

I wanted to know each of this:

If a filesize (eg. the newest/latest one in a directory) is above or below a certain threshold.
If a number of files in a dir is equal, above or below a certain limit.
If the age of a file (eg. the newest/latest one), is above or below a certain age in seconds.

I turned to the batch search/overview sites and looked for inspiration, my findings was

FileSizeComp is an elegant example in batch, but requires you know the filename.
GetDirStats returns number of files, elegantly using dirlist from compact.
I did not find a batch way to get mtime of a file.

So: two problems: I would need a way to find "the latest file" and then pipe that to one of the batch scripts, and I didnt find a ressource kit tool or batch way to get mtime from a file.

So: I made a simple perl script that can handle all of the above. And it also works cross platform.

There was someone who did an mtime (file age) check script in vbscript, i did not use it though.

Identify the process that locks a file on Windows

2008-04-16T07:40:00.007+02:00

I wanted to identify the process that locks a file on Windows, I am sure you know the feeling:

del ft*
The process cannot access the file because it is being used by another process.

There was an execellent article that described how one of my favorite tools procexp.exe from PsTools has a "Find handle or dll" (CTRL+F) that does the trick.

I think process explorer can only see local processes locking a file, it can not see if you have locked using a share for example. I have not tested it.

The same article also describes some possible solutions for command line based, I didnt have a need for that yet though:

Note: There is also a command-line tool named Handle from Windows Sysinternals that can display open handles for any process in the system. See: View Open Handles to a file or folder from the context menu from the Winhelponline.com Blog.
...
Once installed, reboot Windows and use the Oh.exe (Open Handles) command-line tool. For exact parameters, open Help and Support center and type-in OH. The following example shows how to find the Process(es) which have locked the file "INBOX.DBX".
Open a Command Prompt window and type:
oh inbox.dbx >C:\Output.txt

As for the oh.exe method it require reboot after windows 2003 resource kit installation, or you will see:

The system global flag `maintain object type lists' is not enabledfor this system. Please use `oh +otl' to enable it and then reboot.

But it looks very powerfull:

oh - Object handles dump -- built by: dnsrv_dev(v-smgum)
Copyright (c) Microsoft Corporation. All rights reserved.

OH [DUMP_OPTIONS ...]
OH [FLAGS_OPTIONS ...]
OH -c [COMPARE_OPTIONS ...] BEFORE_LOG AFTER_LOG

DUMP_OPTIONS are:

-p N - displays only open handles for process with ID of n. If not
specified perform a system wide dump.
-t TYPENAME - displays only open object names of specified type.
-o FILENAME - specifies the name of the file to write the output to.
-a includes objects with no name.
-s display summary information
-h display stack traces for handles (a process ID must be specified)
-u display only handles with no references in process memory
-v verbose mode (used for debugging oh)
NAME - displays only handles that contain the specified name.

Blog backup reminder

2008-04-14T20:01:00.003+02:00

Very shortly after I started to use this blog as a placeholder for knowledge, I wondered how I could back the blog up.

I didnt get any backup going, so I am happy to get a reminder from a march post on Taosecurity:

Therefore, for the last several months I've been archiving my blogs using BlogBackupOnline.com. I used the free service while in beta, but my storage requirements for this blog exceed their 5 MB "Freemium" limits. Therefore, I ponied up the money for a "Professional" account with 250 MB storage, and the "advertising" provided by this post should help me double that amount to 500 MB.

One of the comments on the post is interesting, I have to try that:

.. also check out http://blogbackupr.com, 100MB free space

Now I am thinking if I can find an open source backup application or script that does RSS backup similar to those services? Then I could cron a backup myself.

Windows command box shell tips

2008-03-13T10:34:00.003+01:00

After so many years with the Windows command line shell, I still learn new stuff every now and then :-)

Today a collegue showed me a feature similar to bash ctrl+R for recalling commands, instead of using up and down arrow:

Type a bit of the command that you know you have used just a while back
Toggle through the commands with F8 ... nice :-)

While I am here, I want to remind myself:

Enable quickedit mode in cmd box options tab: [v] QuickEdit Mode
Increase Screen Buffer Size, Height: 9999
Use doskey /history to get the last commands

More options for third party software updates

2008-02-19T21:16:00.004+01:00

Not long ago I mentioned the Secunia PSI (Personal Software Inspector) as a mean to update your third party software on Windows.

Now I noticed that SANS ISC has a nice article with some more recommendations:

Other options are UpdateStar (Windows), SUMo - Software Update Monitor (Windows), VersionTracker [Pro] (Mac and Windows), RadarSync (Windows), UDC - UpdateChecker (Windows), Belarc Advisor (Windows), and App Update Widget (Mac).

I have not tried any of them yet :-)

The same day they had a really good point, about something that often bothers me on Windows and Mac:

Unprivileged user vs. Administrator: A few third-party Windows software do not show the availability of new updates unless you are running as Administrator.
...
Therefore, the conclusion is that you need to periodically (every day?) login as (or run things as) Administrator to perform periodic tests for new updates. Obviously, this is not practical for end users, so we clearly need to improve the third-party update mechanisms in Windows to be accurate, up-to-date and work smoothly from non-privileged accounts.

Xcacls.vbs directories only and column output truncated

2008-02-12T16:45:00.001+01:00

As i mentioned earlier the xcacls.vbs output is truncated so the information is not fully presented, eg. usernames are cut at 24 characters. This got very annoying, so I was happy to find a solution:
Edit xcacls.vbs line 593, Call PrintMsg( strPackString...
Edit xcacls.vbs line 614, Call AddStringToArray(arraystrACLS,

I changed the two lines to:
Call PrintMsg( strPackString("Type", 8, 1, TRUE) & strPackString("Username", 50, 1, TRUE) & strPackString("Permissions", 42, 1, TRUE) & strPackString("Inheritance", 35, 1, TRUE)) For Each objDACL_Member in objSecDescriptor.DACL

Call AddStringToArray(arraystrACLS, strPackString(strAceType, 8, 1, TRUE) & strPackString(objtrustee.Domain & "\" & objtrustee.Name, 50, 1, TRUE) & strPackString(TempSECString, 42, 1, TRUE) & strPackString(strAceFlags, 35, 1, TRUE),-1) Set objtrustee = Nothing

Now the output is more useful.

The next problem is that I can not get Xcacls.vbs to only work on folders when querying subdirectories. The parameters /s /t does work across subdirs, but it includes files, which is not what I want!

This does not seem possible, i can not find a combination of switches that does travel subdirectories, but only displays directory permissions and not files too. I get output like:
**************************************************************************
Directory: d:\data\file.txt

Permissions:
Type Username Permissions Inheritance
...

So I had to make a small wrapper, to only run XCACLS on a predefined list of dirs, without using any /s /t. This is not scalable at all!

What I would rather like is a script to get a remote dirlisting, where we can check if a filehandle is a dir, and if it is a directory then call xcalcs. I dont have that yet :-)

A better solution is much better.

New remote scanning requirements for PCI compliance

2008-02-12T09:40:00.000+01:00

I have heard there might be new remote scanning requirements for PCI compliance, which assumably means Visa will require a higher level of application scanning that before. Even if it might not be so, it is a good chance to improve the organization IT skills, just as like the original PCI compliance test was a huge improvement.

I am reading parts of the PCI Blog - Compliance Demystified blog, where there are some pointers to documents etc.

In one of the recent PCI Blog newsletters I stumbled upon a some quotes regarding scanning:

Scanning is a snapshot ...
Scanning is diagnostic, not preventative ...
...
In fact SQL Injection, one of the most commonly used methods of
compromise, cannot be detected using scanning.
...
Scanning is a component of the information security program, not a
replacement for it - Scanning can be a useful tool when used as a part of a
robust, well-rounded information security program. Relying on scanning
alone can leave a company dangerously exposed to data compromise. However,
when used in conjunction with timely patch management, strong internal policies
and processes that are actively enforced, data classification and control
practices and other elements of security practice, scanning can provide valuable
insight.

I have to question their statement about "SQL injection can not be found from scanning". As with other vulnerabilities found by scanning, some SQL injections attack vectors can be found. In fact Nessus does a good job of finding some SQL injections, but I have seen Nessus miss SQL injections that was later found by Webinspect. The other points in the newsletter are valid and good to keep in mind!

In the future companies that want to have PCI compliance might be forced by Visa to buy and use either Webinspect or IBM Rational AppScan. Both are very expensive!
The Next Generation of Web Application Scanning
WebInspect
7 is the first and only web application security assessment tool to be
re-architected to thoroughly analyze today's complex web applications built on
emerging Web 2.0 technologies. The new architecture delivers faster scanning
capabilities, broader assessment coverage, and the most accurate results of any
web application scanner available

Open source alternatives for web application scanning tools, that just comes even close to the capabilities of Webinspect and Ration AppScan, would be awesome. Please leave a comment if you have any ideas :-)

Searching your logfiles and your knowledge management sources

2008-02-12T08:01:00.000+01:00

A friend of mine pointed me to Splunk for log file analysis, thanks for that :-)

I havnt had a chance to install and try Splunk, but looking around, Splunk could be the util to combine knowledge management searches with real time event searches from servers. A single point of entry for searching is crucial, but not easy to up and running in the day to day use.

To benefit from a search engine, that engine should be able to reach all the different places that people put knowledge. And it must be able to crawl all file formats, eg. Open office, MS office, excel, pdf etc. We can get the file indexing working from all kinda places, but the hurdle seems to be indexing mailboxes! The example being a public mailbox archive of all the support answers to customers, with many years of useful knowledge! Indexing mailboxes, eg. Lotus Notes, should be possible with enterprise search engines like Google and Yahoo Omnifind.

For logfile analysis, i usually stick with simple tools ala fetchlog, our own grep scripts on centralized syslog servers, and some OSSEC. Other utils I have played with for correlating of information is prelude.

Perhaps Splunk can combine the above (search engine and logfile analysis) into one application?

Splunk provides a free edition, so I will keep it around, in case I get a chance to try it :-) It sure seems worth a try for an enterprise! Of course, being an open source and community fan, I am more biased toward an open source alternative for Splunk? Prelude and OSSEC are both open source free software.

While looking around I stumbled upon an interesting open source site, Softpanorama.org:

Mission and Vision Statement This is a self-education oriented site (see
about for more info) that contains resources for the independent study in
computer science and programming. The latter is the area were open source really
shines: the academic value of open source software (OSS) cannot be
overestimated.

Softpanorama.org has some Splunk entries in their Log Analyzers News:

[Apr. 17, 2006] Splunk Welcome
Splunk is search software that
imitates Google search engine functionality on logs. Can be considered as
the first specialized log search engine. It can correlate some
alerts:
Splunk Splunk User's Guide
Splunk Administrator's Guide

[Feb 16, 2006] Splunk, Nagios partner on open-source systems-monitoring tools
Log file search and indexing software vendor Splunk Inc. announced Tuesday that it will soon add systems
management host, network and service monitoring capabilities to its software
through a partnership with the Nagios open-source project. ...

Not satisfied with your current Version Control System - discussing switching VCS

2008-02-11T13:25:00.000+01:00

At work we are getting increasingly annoyed by the rather old Visual Source Safe we are using. We are going for AccuRev as a replacement. There is an interesting comparison with Subversion. Their Subversion notes might be true, in the sense taht you do need some scripting skills to take full advantage of Subveresion branching and merging. Perhaps this is what you get for the license fee. AccuRev server does on Windows, Mac and Linux, not sure about BSD flavors. It does not come for free:
AccuRev is typically licensed using a named user license model. The
list prices for AccuRev end-user licenses range from $750 to $1,995, depending
on specific products licensed, number of users, and required integrations with
3rd party products (e.g., AccuBridge)

If you want a good reading of version control system discussion and thoughts, I recommend reading the FreeBSD Wiki on the VCS subject. It is very well written, and touches many aspects of version control (also some you probably didnt think about). Of course it is written with reference to the FreeBSD project needs, but if you are a familiar with FreeBSD branches and ports, and working with vendor code for your self, you might get a lot of knowlegde and ideas from reading it. I found it very interesting :-)

In short it is a discussion of open source version control system alternatives, with description of desired and required features, in order to justify the cost of FreeBSD project switching away from CVS. Is similar to our own thoughts on changing version control system here at work.

Most is written by Peter Wemm, who is vouching for Subversion. Here is a snip from Peter's view on why FreeBSD need a new VCS and why Subversion should be the prime target. Should convince you to start reading :-)
Why does my opinion matter? I've been doing this for a while. For the last 13 years, I've been the 'The buck stops here' guy for our repository. I've seen it all. I wrote the rules about what we can and can't do in the repository. I did the hacks to the cvs system to prolong its use for us. I came up with or implemented most of the hair-brained ideas that we live with on a daily basis.
Here are my snips from my reading through all the sections:

Automated or mechanically assisted merging. FreeBSD's development model requires that (unless it's an exceptional circumstance) changes first go in to the HEAD. If they are suitable candidates to go in to stable then they should be merged to the relevant stable branch.
In addition, new features may first be developed on a separate branch, before being merged in to the HEAD.
The VCS should support easy merging of changes from HEAD (or its equivalent) to the stable branches, and from feature branches to HEAD. Merges should also be able to go both ways, and be easily repeatable (e.g., a long lived feature branch may merge changes from HEAD on to the branch several times, and may merge changes from the branch back to HEAD several times)

Branch, Easy & cheap branches (and history-aware merging) and tags to enable parallel lines of development (that is essential for projects like SMPng which have a very big impact on many source files)

SVN Repo Layout: A proposed repository layout if FreeBSD moves to Subversion. This includes a good suggestion of handling Vendor code.

SVN Merging: A walkthrough of merging changes with Subversion and svnmerge.py. This walkthrough of branching and merging is very educational :-)

ACL, Access control: the ability to constrain developers to operating in specific areas of the tree, implement branch-based policy restrictions, as well as to enforce policy such as tagging of commits for developers working outside their normal areas. Implementing these via hooks would not be a regression from what we currently do in CVS.

Offline, Ability to work offline -- like on a plane -- without requiring too much work: not only being able to list differences but also to commit

SVK which brings history-aware merging and distributed features to SVN

There are some really interesting (biased of cource) quotes when it comes to comparing Git and Subversion conversion going from CVS, which are right on, and makes you think:
For us to switch to svn would be an evolutionary step. We could use it
as a better cvs, with the sharp edges fixed. hg and git require more of a
revolution in the way we go about things.

git/hg make it very easy to take stuff offline....Encouraging the
taking of stuff further offline is going in the wrong direction for *us*. If
anything, we need to make it easier for people to get stuff to us and in the
tree in some form.
Linus wrote git to suit his needs for linux. He has one thing going for us that we don't. There is a large cult of personality surrounding Linus. There is intense pressure to "validate" your work by getting it approved (directly or by proxy) by Linus. On the other hand, we already have problems extracting work from people. We can't assume that we'll get the same inward flow that Linus gets.
From http://lwn.net/Articles/246381/ - there are some choice quotes. The topic is the problems the KDE folks had making git work for them.
We're not Linux. A good number of our best supporters stick with us because we're a coherent tree and not like linux' chaos.
Why do you seem to be pushing subversion?It's because I am. I think the whole hg/git thing is a distraction.
it works the same way we've become accustomed to cvs working. Except without most of the silly problems/restrictions.
there are a huge bunch of tools out there to talk to svn. Things like svnsync (cvsup for svn repository replication) are out there.
We can use live changeset based exporting to export the tree to cvs to maintain HEAD and RELENG_* branches. Our end users will be able to keep doing exactly what they've always been doing for getting their "fix" of freebsd.
svk, as an optional add-on, gets you the ability to have a private playground, in spite of my encouragement to work on the public servers.
Notes on Git Conversion: Why git is interesting to FreeBSD, is also very educating. From the little bit of Git reading that I have done, it seems to me that Git gives abilities to hide development cycles, not something I would appreciate in the projects I participate in. Some Git quotes:

git is distributed
Now, you can commit as you develop, then test, then push. If
you find things in your testing that are wrong, you can commit fixes before
pushing, or even go back and edit your local history to erase your mistakes,
making you look even more ninja than you really are.
You can also push your
changes up to a personal repository for others to access. They can merge it to a
personal tree of their own, do repeated merges all sorts of directions, and have
it just Do The Right Thing.

I am a fan of Subversion, and it works on many platforms. So far Subversion has fitted all my needs for version control, automation, documentation, management etc!

After reading the above articles I am even more convinced Subversion will continue to meet my needs, so I am not changing :-) SVK is something for me to try though. And AccuRev might prove useful for the enterprise, we will see.

Query MSSQL from perl

2008-02-06T13:46:00.000+01:00

I mentioned how to connect to MSSQL from batch, eg. using osql.exe, but today I wanted to do the same from Perl.

There are many samples on Google, using Win32::OLE or Win32::ODBC. Usually finding the right connection string is the hurdle.

For the ODBC connection strings it can look like this:

$DSN = 'driver={SQL Server};Server=$hostname\\$instance;database=$db;uid=$u;pwd=$p;';
if (!($db = new Win32::ODBC($DSN))){ die "Error: " . Win32::ODBC::Error() . "\n"; }

For Win32::OLE connection string with password can look like this:
my $ConnStr="Provider=SQLOLEDB;Initial Catalog=$db;Data Source=$server;User ID=$u;Password=$p;Network Library=DBMSSOCN";

But I really want to avoid the user and password in scripts. So for Win32::OLE connection string integrated security, without password, can look like this:
my $ConnStr="Provider=SQLOLEDB;Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=$d;Data Source=$s;use Procedure for Prepare=0;Connect Timeout=4;Trusted_Connection=Yes";
# Provider=SQLOLEDB.1 or Provider=SQLNCLI.1

Testing connection, create a query, execute it, and work with is pretty straight forward:
my $Conn = Win32::OLE-> new('ADODB.Connection');
$Conn-> Open($ConnStr);
my $err = Win32::OLE::LastError();
if (not $err eq "0") { print"FATAL: no connection, OLE error 0x%08x: $err\n"; exit; } else { print"Connected OK\n"; }
my $Statement = "select servername from servertable where x = 0 AND id = 11";
if(! ($RS = $Conn->Execute($Statement)))
{ print Win32::OLE->LastError() ; exit; }
while (! $RS->EOF) {
$servername= $RS->Fields(0)->value;
print"servername is: $servername\n";
$RS->MoveNext; }
$RS->Close;
$Conn->Close;

Just for future reference the ODBC SQL update code could look like this:
$SqlStatement = "insert into dbo.MyTable values (\'$var1\',$var2,$number,getdate())";
if ($db->Sql($SqlStatement)){ print "Error: " . $db->Error() . "\n"; $db->Close(); exit; }

Windows IT Pro dec07 notes

2008-01-31T20:18:00.002+01:00

It was a while since I read december 2007 Windows IT Pro issue, but here goes some notes from stuff I found particular useful, or otherwise hard to remember :-)

PDF utilities, similar to PDFcreator (that does not work on Vista), and working on Vista: CutePDF and PDFTools. I use PDFcreator all the time, but I dont use Vista just yet, but this is good to keep in mind! Some quotes from PDFTools features:

• encrypt a PDF file by assigning it a password
• create a protection-free version of encrypted PDF file
• create a PDF file by joining multiple PDF files
• split a PDF file in multiple ways, such as splitting each page to new PDF file and splitting a file after a given page number
• arrange pages in a PDF file
• overlay text or an image over a PDF file
• convert an XML file into a PDF file
Note that you need Java Runtime Environment/Java Development Kit (JRE/JDK) 1.4 or later to use PDFTools.

It was interesting to read "Are IT Pros Steering Their Children Away From IT" and "A Good Career For Your Kids", as since I recently became a dad I have actually been thinking about this several times!

I am certainly not recommending IT business to just anyone. I like to think of IT as being a "call", a desire, for helping people, deliver results, services and service to people, a call to engineer stable and innovative IT systems. For my daughter I will encourage her to do what ever she likes, and if that is IT, that is OK. I think an IT developer, administrator or project manager can be a happy job, but you have to be very aware of the all the factors. This is a hard topic to just put down some notes on, I definately have to give it some more thoughts and its own post, its really interesting and worth dicussing with wife, family and friends!

As you probably know Windows 2008 will have the Server Core option, which has a very limited GUI. This will be exciting to use, and undoubtly we will have to get used to many new command line utils and new/better usage of existing ones. One of the new tools we will get used to is the script SCRegEdit (Server Core RegEdit). Although regedit is a gui that will work in Server Core, SCRegEdit will help edit many registry keys, eg.:
scregedit /ar 0 enables remote desktop
scregedit /au 4 will download and install updates
scregedit /cs will allow Pre-Vista clients to connect with terminal services
... check out Microsofts Server Core guide, there are good tips:

Managing a Server Core installation: Overview
...
The script is located in the \Windows\System32 folder of a server running a Server Core installation. At a command prompt, open the folder, and then use the following command to display the usage instructions for the previous options:

cscript scregedit.wsf /?

Note:
You can use this command with the /cli option to display a list of common command-line tools and their usage.

Some VMware Server and VMware Workstation differences. So far I have managed fine with Server version. ESX will be next, the Workstation is never considered! But anyways interesting:

Price, VMware Server is free :-)
Server is service, Workstation is a desktop application
Multiuser access for Server, not for Workstation. Server has a webinterface too!
Workstation supports virtuals machines (VM) up to 8 GB RAM, Server only 3.6 GB RAM.
Server can have only one snapshot, Workstation can have many.
You can clone a Workstation virtual machine. In Server copying a VM is a manual process, but works fine.
Workstation lets you manage several VMs in teams, eg. to have certain VMs startup before others. Server dont have VM Teams.
Drag and drop objects from your desktop to the Workstation VMs. Server can not.
You can capture an .avi movie of all activity in a Workstation VM.

So it was another great Windows IT Pro issue :-)

UPDATE: It was brought to my attention that VMWare workstation can be set up as a service [http://blogs.techrepublic.com.com/datacenter/?p=429&tag=nl.e101]

Query MSSQL from batch

2008-01-31T10:56:00.000+01:00

Sometimes you want to perform the same batch task on several servers. For that I need a list of the servers that will need some job done. So I want to get the serverlist from the serverdatabase, instead of hardcoding the scripts. The most obvious would probably be using VBscript, but in this case turned to osql.exe for a quick solution:

set sqlbinary="\\someserver\c$\Program Files\Microsoft SQL Server\80\Tools\Binn\osql.exe"
set sqlserverinstance=HOSTNAME\INSTANCE
FOR /F "usebackq" %%A IN (`tempfile`) DO set sqltmp=%%A
set sqltmp=%sqltmp:/=\%
if exist %sqltmp% del %sqltmp%
echo Creating sql inputfile : %sqltmp%
echo set nocount on > %sqltmp%
echo select ServerName+^'::^'+ServerDesc >> %sqltmp%
echo from ServerTable where ServerType = 1 AND ServerGroup = 11 >> %sqltmp%
echo go >> %sqltmp%
%sqlbin% -d ServerDatabase -i %sqltmp% -n -E -S %sqlserverinstance% | egrep "^ [sS][0-9]" | sort | sed 's/^[ \t]*//'

Now I have a list with servernames and descriptions, which I can pipe to a .txt file or perform something on each :)

By the way, I stumbled upon an awesome Batch FAQ, really old, but with some very good points and links to more info. Here are some quotes:

*** How do I perform if-then-else in batch?

if not .%1==.help goto else
rem then commands here
goto endif
:else
rem else conditions here
:endif

...

*** What do all those }{ and $ things mean?

They're uniquely named temp files or variable names. It is
desirable to make the filenames as weird as possible to avoid
overwriting files that happen to have the same name. Also,
confusion is found in spacing and where the redirection
characters are, these all write "hey!" to a temp file...

echo>[myfile] hey!
>$$$tmp$$.$ echo hey!
echo hey! > tempfile

...

*** Utility programs for batch files

Batch simply wasn't designed to do the kinds of things users
want to do, although us batch hackers ignore this and try to
do them anyway. Batch input routines are especially kludgy
and incompatible, often it's easier to just use a utility
designed for the purpose and avoid the hassle.

SENVAR by Ed Schwartz makes it very easy to set an environment
variable to standard-input...

senvar evar - input from keyboard
program senvar evar > nul - input from program
senvar evar <> nul - input from file

SENVAR is at http://www.infionline.net/~wtnewton/batch/senvar.txt

The shareware XSET program by Marc Stern has many extra options,
like reading a file from a specific column and line number...

xset /mid 6 2 /line 3 evar <> nul

XSET is at http://members.tripod.com/~marcstern/xset.htm

ASET by Richard Breuer, free, makes mathematical operations as
easy as ASET result=2+2, functions for math, string handling,
file/kb input and more. File aset10.zip at Simtel.

Strings by Douglas Boling, free, provides commands for string
handling, modifying memory and master environment, reading files,
math, keyboard input and more. File string25.zip at Simtel.

Many more useful batch utilities can be found at...
Garbo: http://garbo.uwasa.fi/pc/batchutil.html
SimTel: ftp://ftp.simtel.net/pub/simtelnet/msdos/batchutl/

Playing with cmd, start and exit commands and parameters

2008-01-30T16:15:00.000+01:00

If you are playing with Windows batch files you are probably using cmd parameters, such as /k to keep cmd box, or /c to close it after command completes, eg:
psexec \\server -e cmd /c "reg import d:\registry_setting.reg"

And similar, you are probably using "exit /b 1" to set errorlevel (returncode) of your script to 1 if it somehow failed.

I havnt used "start" before, but i had a bunch of scheduled tasks and one of those is running every minute, so I figured I would use "start /MIN". This workaround came to mind, because I have no idea how to make sure a tasks is running in session 0 for example, so the repeating task (every minute) can popup with stuff it is doing.. very annoying!

So I added "start /MIN" before my .bat script, but that was not enough. Running the scheduled task would not really start the script. So i added "cmd /C start /MIN somescript.bat", ugly but it worked! Now the scheduled task is minimized on every run.

I noticed that the start command creates its own "cmd /K" process, so my solution results in a process command line like this: "cmd /K somescript.bat". This means that because I am starting the somescript.bat with "start", I now have to add a trailing "exit" in the somescript.bat. Also ugly, but it works.

Now the weird thing I have been puzzled about is a bunch of cmd.exe processes hanging! Using procexp (part of pstools) I can see they are all started from within a Batch control system by running command "start anotherscript.bat". But the anotherscript.bat *does* actually have an exit at the end, so it seems strange that it is hanging. Perhaps it is a hickup in the batch control system!

I can not reproduce a hanging cmd.exe exit command, but I did manage somehow, with a bunch of start, cmd, exit, exit /b 1, etc etc, to create a hanging cmd.exe, where exit command would NOT complete! I dont know how, but in process explorer (procexp), I could see the cmd that was hanging. What could be happening is that exit hangs it self if a child process has disappeared. From the procexp I can not bring window for hanging cmd.exe pid 4696 to front. And then exit command inside cmd.exe pid 4448 is hanging for ever! It did not help to kill 4696 manually, exit of 4448 is still hanging! I had to kill 4448 manually, very annoying!

I suspect it being something weird with start and exit usage, but I am not sure. The exit /? puzzles me, and i am always using exit /B 1 instead of just exit 1. Maybe thats wrong?
exit /?
Quits the CMD.EXE program (command
interpreter) or the current batch
script.

EXIT [/B] [exitCode]

/B specifies to exit the current batch script instead of
CMD.EXE. If executed from outside a batch script, it
will quit CMD.EXE

exitCode
specifies a numeric number. if /B is specified, sets
ERRORLEVEL that number.
If quitting CMD.EXE, sets the process
exit code with that number.

Windows users and groups information

2008-01-29T14:51:00.000+01:00

Being part of a Windows administrator group, responsible for a bunch of Windows server, where there is more than one administrator can be quite challenging!

We have a bunch of scripts that does some automatic documentation of:

Server shares
Server scheduled tasks
Server services with sc.exe
Ntfs permissions of selected directories.

Now I want add a script for documentation of the server users and groups!

Here is my first thoughts of what I would like:

1) given a username, script should return:
show group membership
show username details

2) given a groupname, script must give:
show members
show username details for each member

3) given a servername, return list of:
local users and run 1) for each username
local groupnames and run 2) for each groupname

I did some Google searches:

enumerate group memberslist of members in a local group, eg. who is member
of "administrators"
backup and recovery of windows users and groups
list of users and groups on windows server
enumerate local users and their membership
enumerate windows users with wmi

I ended up with a simple vbscript that combines a good userinfomation binary with some user and group info vbscript code. The output from the script is text, easily diffable, so changes can quickly be spottet.

Someone else surely should have cooked up something smart, as this task seems like something many administrators would appreciate. If you know of such script or application, please leave a comment :-)

A thing that puzzled me for a while was how to get output from the binary into the same STDOUT where I would be starting my script with cscript.exe listusersandgroups.wsf. This was needed as I want to pipe script output to a text file for version control commit and change management :-) So this was easily worked around like this:
set objWshShell = CreateObject("WScript.Shell")
set objWshShell = objWshShell.Exec(strCommand)
Do While objWshShell.StdOut.AtEndOfStream<>True
' running a file from inside vbscript and get output in same command window
strLine=objWshShell.StdOut.ReadLine
WScript.Echo strLine
Loop

The usual way I have started programs from inside VBscript, would be to have them hidden, similar to this:
set objWshShell = objWshShell.Exec(strCommand)
intRC = objWshShell.Run(StrCommand, 0, TRUE)
' parm 1 = command line
' parm 2 = window style (1 = normal, 0 = hidden)
' parm 3 = if true, waits for command
If intRC <> 0 Then ...
' and destroy it properly:
if isObject(objWshShell) then set objWshShell = nothing

Read more about the normal .Run method.

Software Inspector for personal Windows package management

2008-01-24T08:34:00.000+01:00

Windows package management is not an easy task, often left only to WSUS server, WPKG, group policies or simply the individual applications automatic updates.

Leaving updates to only the applications and users themselves obviously is not good enough! Although this has gotten better the last years, some sort of action and verification is needed.

For example the Java Runtime Environment does not deinstall old versions when new versions are installed. Probably as a service for you, so your old java applications can decide which one they want to use, avoiding problems with incompatibility. But this update strategy also leaves a hole for crackers to potentially abuse!

I did mention installing and updating Windows applications with win-get, but it probably is not a option for anyone else than the tech geek at home.

How much package management Microsofts new NAP service can handle is unknown to me. What I have read so far sounds like it can do limited checks, such as if Windows hotfixes are applied and Firewall is on. I need hands on to know more I guess.

Well, here is something worth trying Software Inspector from Secunia:

Online version:
http://secunia.com/software_inspector/
Secunia Software Inspector
Feature Overview - The Secunia Software Inspector:
* Detects insecure versions of applications installed
* Verifies that all Microsoft patches are applied
* Assists you in updating your system and applications
* Runs through your browser. No installation or download is required.

Offline, for installation on your PC:
https://psi.secunia.com/
Version: 0.9.0.0 / Size: 444,892 bytes / Changelog
The Secunia PSI is only free for private individuals
The Secunia PSI is available free of charge.
Secure your PC. Patch your applications. Be proactive.
Scan for Insecure and End-of-Life applications.
Track your patch-performance week by week.
Direct and easy access to security patches.
Detect more than 300,000 unique application versions

For work, it would be perfect to have a central Software Inspector server (ala WSUS server) which has info of all computers where an agent was installed. Great for reporting and verification of your Windows package management tools actually does its thing as you are expecting!

Blogger.com post editor

2008-01-23T10:29:00.000+01:00

In my recent batch post I had to use the pipe symbol (|) in some example listings. It turned out that the blogger.com editor removes the pipe symbols when switching from HTML to wysiwyg editing.

Worse is that if you are in Wysiwyg mode and looking at the pipe symbol in your text, it will disapear from your post if you publish from there!!

I tried to escape the pipe symbol with \ and other ways, but it simply disappeared!

I looked at wikipedia "vertical bar article" and found the ASCII value for the pipe symbol:

ASCII
decimal (base-10): 124, or hexadecimal (base-16): 7C

Then i looked for HTML article on how to escape symbols, similar to the &. It was an XML guide that really putted it simple:

XML has the same syntax as HTML for escape symbols like "&"="&", "<"="<", ">"=">", ascii(nnn)=&#nnn;", etc.

The solution was write all pipe symbols (|) as "&#124" from the HTML view of the editor, then not switching back to GUI, simply publising from HTML view at once:

For this article it looked like this when I pushed publish: (notice how the text for < is written):

And the result on blogspot.com was as i wanted:

Batch script userinput checking

2008-01-23T09:31:00.000+01:00

I have to run a script with the runas command, but since the script is running commands toward several servers there is the danger of locking out the runas user if the password given is wrong. Simply because runas does not verify the password, it just executes the commands.

So to avoid problems I would like to ask the user for the password, verify the password, and only actually run the runas command if the password is as expected.

At a first glance this sounded good, I just had to put in the checksum of the expected userinput in the script, then calculate the checksum of the userinput, and compare the two inside the script.

At second thought this solution was not really acceptable, because if the users password changes, you would have to update the script! Not very robust or elegant. So instead a colleague pointed out the obvious, which of course is to check errorlevel of a single run of runas. That should not lock out the user:
runas /user:domain\username net >nul 2>&1
if not errorlevel 0 (set status=failure & goto exiting)

To make any of above approaches work, we needed a method of getting users input, and a method of sending that input to runas.

Getting the users input in a .batch file was solved by using a special .com file:
echo hP1X500P[PZBBBfh#b##fXf-V@`$fPf]f3/f1/5++u5>%inputfile%
echo Enter a string (it will not echo here):
for /f "tokens=*" %%i in ('%inputfile%') do set userinput=%%i
if "%userinput%"=="" (set status=stringempty & goto exiting)
if "%userinput%"=="^C" (set status=stringcancel & goto exiting)

Sending the %userinput% content back to runas could not be done with a redirection like <, so a mini vbscript for pasting a string was made: Set oArgs=wscript.Arguments
WScript.sleep(1000)
Set WshShell = CreateObject("WScript.Shell")
WshShell.SendKeys oArgs(0)&VBCRLF

Others have been discussing how to pipe passwords into runas, but i dont want to use the approaches described.

So problem was solved without using hardcodet md5 or sha256 checksums! But just to have the hardcoded approach for optional other use, here is how it was done:
FOR /F "usebackq" %%A IN (`echo %userinput% ^| md5deep.exe`) DO set md5hash=%%A
echo debug: md5hash of userinput is: %md5hash%
if "%md5hash%" == "77e2d91aa21a4158d889fb9836f38288" (set status=ok_string_is_hej & goto dosomething)
if "%md5hash%" == "291013bf3a3c543625a2777073f91799" (set status=ok_string_is_password & goto dosomething)

In the above i used md5deep to calculate a checksum of the batch string, and it could easily have been sha256deep.exe which is part of md5deep package.

I would have used Microsofts own Windows checksum util FCIV (FILE CHECKSUM INTEGRITY VERIFIER), but there are at least 2 problems with the current version 2.05:

First, fciv.exe does not take input from STDIN, which means you can not run like this:
echo foobar | fciv.exe

You have to echo into a file and then calculate the checksum:
echo foobar > foobar.txt
fciv.exe foobar.txt
//
// File Checksum Integrity Verifier version 2.05.
//
5e963b88334c3c4487572cce68496989 foobar.txt

So I used md5deep package, which actually does calculate checksum of input from stdin, useful for strings:
echo foobar | sha256deep.exe
791132eb55910a285d5bfeae94b49ead8d5184d7ecf70bccdeafd0e456c2916d
echo foobar | md5deep.exe
5e963b88334c3c4487572cce68496989

The second problem with fciv.exe is the output: it is too verbose! I would need only the checksum so I can put it into a variable. So md5deep it is!

The trick to actually get the output from external md5deep command into a batch variable, is to use a for loop, and escape the pipe () with a ^ instead of a \ which was what I tried first.

Apparently the hat (^) is the dos char for escaping, eg. used if you want to echo the following characters from a .batch file into another file: ^@, ^> and ^&. For example:
echo ^@echo off > c:\tempscript.bat
echo dir c:\ ^> c:\dirlist.txt >> c:\tempscript.bat
echo del c:\tempfile.log /F /Q >> c:\tempscript.bat
echo exit >> c:\tempscript.bat

Another md5 sum checker, built in java (source available), which can check a dir and subdirs, like md5deep.exe can be found here.

Secure data handling - the power to raid!

2008-01-21T18:12:00.000+01:00

When I read that another laptop with personal data was missing, in Britain again, I remembered reading TaoSecurity predictions for 2008, there was something about an initiative about giving power to raid Governmen departments. It looks like that initiative really needs to get going!

I really hope this power is moving toward companies as well. If you handle personal data, you should be suspect to unexpected tests! Much similar to internal revision and penetration testing.

But should the results from such a new "data protection department" be public, similar to the smiley for restaurents and cafes? If so, should it be public before or after problems are fixed? Like full disclosure, it has it pros and cons. But customers would know if a particular company took data security and handling seriously! Maybe if something is a bit more expensive, but has a better "data handling smiley", I as a customer could make the choice myself. Opposed to now, I really dont know how good or bad companies handle my personal data!

The Information Commissioner’s Office (ICO), which polices the security of the nation’s data, is to be given the power to raid Government departments suspected of breaching protection laws.

The move, announced today by Gordon Brown, comes in response to the loss by HM Revenue & Customs (HMRC) of personal details of some 25 million Britons. The Prime Minister said the ICO would be given extra powers to carry out “spot checks” of government departments.

He added: "For some time I have been pressing the government to give my Office the power to audit and inspect organisations that process people’s personal information without first having to get their consent."

Defensible Network Architecture 2.0

2008-01-11T23:10:00.000+01:00

Taosecurity starts off 2008 with another great "tasks/topics to consider for IT administrations", similar to "IT security - determine your score of the game".

Once again, any administrator should read his blog :-)

This time its a version 2.0 of how to have a network architecture that gives you "the best chance to resist intrusion, since perfect intrusion prevention is impossible":

A Defensible Network Architecture is an information architecture that is:

Monitored. The easiest and cheapest way to begin developing DNA on an existing enterprise is to deploy Network Security Monitoring sensors capturing session data (at an absolute minimum), full content data (if you can get it), and statistical data. If you can access other data sources, like firewall/router/IPS/DNS/proxy/whatever logs, begin working that angle too. Save the tougher data types (those that require reconfiguring assets and buying mammoth databases) until much later. This needs to be a quick win with the data in the hands of a small, centralized group. You should always start by monitoring first, as Bruce Schneier proclaimed so well in 2001.

Inventoried. This means knowing what you host on your network. If you've started monitoring you can acquire a lot of this information passively. This is new to DNA 2.0 because I assumed it would be already done previously. Fat chance!

Controlled. Now that you know how your network is operating and what is on it, you can start implementing network-based controls. Take this anyway you wish -- ingress filtering, egress filtering, network admission control, network access control, proxy connections, and so on. The idea is you transition from an "anything goes" network to one where the activity is authorized in advance, if possible. This step marks the first time where stakeholders might start complaining.

Claimed. Now you are really going to reach out and touch a stakeholder. Claimed means identifying asset owners and developing policies, procedures, and plans for the operation of that asset. Feel free to swap this item with the previous. In my experience it is usually easier to start introducing control before making people take ownership of systems. This step is a prerequisite for performing incident response. We can detect intrusions in the first step. We can only work with an asset owner to respond when we know who owns the asset and how we can contain and recover it.

Minimized. This step is the first to directly impact the configuration and posture of assets. Here we work with stakeholders to reduce the attack surface of their network devices. You can apply this idea to clients, servers, applications, network links, and so on. By reducing attack surface area you improve your ability to perform all of the other steps, but you can't really implement minimization until you know who owns what.

Assessed. This is a vulnerability assessment process to identify weaknesses in assets. You could easily place this step before minimization. Some might argue that it pays to begin with an assessment, but the first question is going to be: "What do we assess?" I think it might be easier to start disabling unnecessary services first, but you may not know what's running on the machines without assessing them. Also consider performing an adversary simulation to test your overall security operations. Assessment is the step where you decide if what you've done so far is making any difference.

Current. Current means keeping your assets configured and patched such that they can resist known attacks by addressing known vulnerabilities. It's easy to disable functionality no one needs. However, upgrades can sometimes break applications. That's why this step is last. It's the final piece in DNA 2.0.

Event log ID to description and vice versa

2008-01-11T20:40:00.000+01:00

Where do you go for event log id information? Google it - as anything else :-) Today i wanted to know the event ID for a Windows 2003 server rebooting, how do i find that?

Besides google it, I could look in an event log around the time of a reboot for a server.

I can not search it on Microsofts Events and Errors Message Center. It is useful if you have the event id, but not really for free text search.

I came closer when looking at Ultimate Windows Security website, but the lists are not complete and I didnt see an option for search.

512
All Versions

Windows NT is starting up

513
Win2003
XP

Windows NT is shutting downI

I thought EventID.net would be the place, but i can also only look up know ID numbers. They do require a registration fee for the more exotic search options, so perhaps I need to go there, I dont know. Besides that looking up info about eg. event id 513 gives really useful information:

Source	Security
Type	Success Audit
Description	Windows NT is shutting down. All logon sessions will be terminated by this shutdown.
English please!	Request a translation of the event description in plain English! An example of "English please" is available here.
Details	Comments and links for event id 513 from source Security

I will give the 3 scripts evtstats.pl/lsevt2.pl/lsevt.pl from Windows Security Analysis a try, as with that i can run a query toward a Windows 2003 server and grep for the word reboot. Perhaps this is the best way :-)

Other than this, i am not sure how to find this information, besides Googling of course :-)

Oh, while I am at it, i will leave a link to Stephen Bunting guide of repairing event log files.

Book review: Windows Forensic Analysis

2008-01-11T19:31:00.000+01:00

I finally had a chance to finish reading Windows Forensic Analysis. From the start I was happy with the way the book is written, and I give it 5 of 5 possible. I am not working with forensics in my daily administration work, but I learned a lot about security methods and tools in general, something that can be of great help for normal administration too.

The book is flooded with tools worth trying, and with examples usage. The examples illustrates the topic really well, and the tools are perfect to get started on your own.

In addition to tool suggestions the author provides the reader with many perl scripts, which further helps in better understanding of the topic, and makes you want to learn more! The scripts are also simply useful and saves you a lot of time when you want to try the stuff discussed in the book. I just loooved the scripts!

I consider myself an okay experienced Windows administrator (since 1999), and many of the topics was not new. But I liked another explanation of the topics, and I also learned a lot of totally new stuff.

I recommend all Windows administrators reading this book! And dont forget to read Richard Bejtlichs review (TaoSecurity), he knows how to wrote really useful reviews!

Apache, disable debugging functions

2008-01-07T10:45:00.000+01:00

I want to disable debugging function TRACE as recommended by Nessus rule 11213.

The Nessus rule has very precise guide for disabling:

Add the following lines for each virtual host in your configuration file :

RewriteEngine on
RewriteCond %{REQUEST_METHOD} ^(TRACETRACK)
RewriteRule .* - [F]

Alternatively, note that Apache versions 1.3.34, 2.0.55, and 2.2
support disabling the TRACE method natively via the 'TraceEnable'
directive.

So in my Apache main config file I putted TraceEnable off.

I will check the Nessus scan results after these changes.
If you can not wait for that, you can use telnet to check if TRACE is actually off:
telnet your.server 80
TRACE / HTTP/1.1
Host: a.valid.hostname
sometext

Apache, restrict connections to SSL 3.0 and ensure strong encryption

2008-01-07T10:08:00.000+01:00

For my Apache I wanted disable SSL 2.0 as recommended by Nessus rules 20007 , and to disable weak SSL ciphers as recommended by Nessus rule 26928.

When looking around I found a nice description at Novell:

Use only High and Medium security cipher suites, such as RC4 and RSA.
Remove from consideration any ciphers that do not authenticate, such as Anonymous Diffie-Hellman (ADH) ciphers.
Use SSL 3.0, and disable SSL 2.0.
Disable the Low, Export, and Null cipher suites.

So for Apache I did the following:
Open the /etc/httpd/conf/httpd.conf file in a text editor, then locate the SSLCipherSuite directive in the Virtual Hosts section:
SSLCipherSuite ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL

Modify the plus (+) to a minus (-) in front of the ciphers you want to disable and make sure there is a ! (not) before ADH:
SSLCipherSuite ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:-LOW:-SSLv2:-EXP:-eNULL

I will check Nessus scan output after the changes.

Fetchlog alternatives for Windows and 2003/2000/NT resource kit tools

2008-01-04T21:12:00.000+01:00

I am looking for an alternative to the simple fetchlog util on unix, which tails a file and has a bookmark of how far it has checked in the file. When a string is found, i can do action, such as launch script, send mail or restart services. Works well for some simple purposes :-)

I am not looking for full blown log analysis, such as OSSEC which I really like though.

I havnt found anything that I really want to use, but here is my list of things to try:

WinTail. $49.95, with basics such as send notify mails, but can it run a script, eg. a restart of a service? That is really often needed. 30 day trial, worth a try I guess.

Some more simple tail tools:
Tail4Win. Also 30 day trial, $45, looks similar to normal tail, there seems no notify option.
MakeLogic Tail. Freeware, tails more than one file, requires JRE 5.0, has no notifications it seems.
tail.exe. Windows 2003 resource kit (see below), does not tail more than one file!
Tail Ace. Freeware, multiple logfiles, but no notifications, java based, requires JRE 6.0.
Tail XP. Freeware. Tails more than one file, but you can not see difference, and it is showing in a gui which can not be piped to a grep for example. Only takes one file from commandline -f argument, and still opens it in a gui. No notifications.

None of the above really meets what I at least need of a fetchlog tool. If I can not find a fetchlog alike tool, I would at least need a grep tool that can tail -f more than one file, and from commandline. And output must be possible to send to STDOUT for more processing and use in scripts, eg. based on errorlevel. So far I havnt found anything that does this!

Some of the more full blown tools I stumbled upon was:
http://www.xpolog.com/home/products/xpologCenter.jsp

A comprehensive loganalyzer tool overview is available at download32.com, but it is all the stuff, not just what I am looking for, this tail -f specifics overview is not much better. Perhaps using a unix tools on Windows would be better than using the simple tools above, because those tools can be piped into other commands, which i really need.

Of course a normal tail without -f wont do, but since it has so nice batch example code here it is (for more of the same, take a look here and here):

@echo off
if {%1}=={} @echo FileName parameter requied.&goto :EOF
if not exist %1 @echo %1 does NOT exist.&goto :EOF
setlocal
set file=%1
set /a number=10
if not {%2}=={} set /a number=%2
for /f %%i in ('find /v /c "" ^< %file%') do set /a lines=%%i @echo %lines% lines in file %file%. if %number% GEQ %lines% set /a start=0&goto console set /a start=%lines% - %number% :console more /e +%start% %file% endlocal

It was a surprise to me that the Windows 2003 resource kit free tools has a tail.exe, of course it is not enough for notifications and actions, and it can not tail more than one file! For completeness, here is the Windows 2003 resource kit tool list:

Acctinfo.dll (documented in Readme.htm)
Adlb.exe: Active Directory Load Balancing Tool
Admx.msi: ADM File Parser
Atmarp.exe: Windows ATM ARP Server Information Tool
Atmlane.exe: Windows ATM LAN Emulation Client Information
Autoexnt.exe: AutoExNT Service
Cdburn.exe: ISO CD-ROM Burner Tool
Checkrepl.vbs: Check Replication
Chklnks.exe: Link Check Wizard
Chknic.exe: Network Interface Card Compliance Tool for Network Load Balancing
Cleanspl.exe: Spooler Cleaner
Clearmem.exe: Clear Memory
Clusdiag.msi: Cluster Diagnostics and Verification Tool
Clusfileport.dll: Cluster Print File Port
Clusterrecovery.exe: Server Cluster Recovery Utility
Cmdhere.inf: Command Here
Cmgetcer.dll: Connection Manager Certificate Deployment Tool
Compress.exe: Compress Files
Confdisk.exe: Disk Configuration Tool
Consume.exe: Memory Consumers Tool
Creatfil.exe: Create File
Csccmd.exe: Client-Side Caching Command-Line Options
Custreasonedit.exe: Custom Reason Editor (documented in Readme.htm)
Delprof.exe: User Profile Deletion Utility
Dh.exe: Display Heap
Diskraid.exe: RAID Configuration Tool
Diskuse.exe: User Disk Usage Tool
Dnsdiag.exe: SMTP DNS Diagnostic Tool (documented in Readme.htm)
Dumpfsmos.cmd: Dump FSMO Roles
Dvdburn.exe: ISO DVD Burner Tool
Empty.exe: Free Working Set Tool
Eventcombmt.exe: Check Replication
Fcopy.exe: File Copy Utility for Message Queuing
Frsflags.vbs
Getcm.exe: Connection Manager Profile Update
Gpmonitor.exe: Group Policy Monitor
Gpotool.exe: Group Policy Objects
Hlscan.exe: Hard Link Display Tool
Ifilttst.exe: IFilter Test Suite
Ifmember.exe: User Membership Tool
Inetesc.adm: Internet Explorer Enhanced Security Configuration
Iniman.exe: Initialization Files Manipulation Tool
Instcm.exe: Install Connection Manager Profile
Instsrv.exe: Service Installer
Intfiltr.exe: Interrupt Affinity Tool
Kerbtray.exe: Kerberos Tray
Kernrate.exe: Kernel Profiling Tool
Klist.exe: Kerberos List
Krt.exe: Certification Authority Key Recovery
Lbridge.cmd: L-Bridge
Linkd.exe
Linkspeed.exe: Link Speed
List.exe: List Text File Tool
Lockoutstatus.exe: Account Lockout Status (documented in Readme.htm)
Logtime.exe
Lsreport.exe: Terminal Services Licensing Reporter
Lsview.exe: Terminal Services License Server Viewer
Mcast.exe: Multicast Packet Tool
Memmonitor.exe: Memory Monitor
Memtriage.exe: Resource Leak Triage Tool
Mibcc.exe: SNMP MIB Compiler
Moveuser.exe: Move Users
Mscep.dll: Certificate Services Add-on for Simple Certificate Enrollment Protocol
Nlsinfo.exe: Locale Information Tool
Now.exe: STDOUT Current Date and Time
Ntimer.exe: Windows Program Timer
Ntrights.exe
Oh.exe: Open Handles
Oleview.exe: OLE/COM Object Viewer
Pathman.exe: Path Manager
Permcopy.exe: Share Permissions Copy
Perms.exe: User File Permissions Tool
Pfmon.exe: Page Fault Monitor
Pkiview.msc: PKI Health Tool
Pmon.exe: Process Resource Monitor
Printdriverinfo.exe: Drivers Source
Prnadmin.dll: Printer Administration Objects
Qgrep.exe
Qtcp.exe: QoS Time Stamp
Queryad.vbs: Query Active Directory
Rassrvmon.exe: RAS Server Monitor
Rcontrolad.exe: Active Directory Remote Control Add-On
Regini.exe: Registry Change by Script
Regview.exe (documented in Readme.htm)
Remapkey.exe: Remap Windows Keyboard Layout
Robocopy.exe: Robust File Copy Utility
Rpccfg.exe: RPC Configuration Tool
Rpcdump.exe
Rpcping.exe
RPing: RPC Connectivity Verification Tool
Rqc.exe: Remote Access Quarantine Client
Rqs.exe: Remote Access Quarantine Agent
Setprinter.exe: Spooler Configuration Tool
Showacls.exe
Showperf.exe: Performance Data Block Dump Utility
Showpriv.exe: Show Privilege
Sleep.exe: Batch File Wait
Sonar.exe: FRS Status Viewer
Splinfo.exe: Print Spooler Information
Srvany.exe: Applications as Services Utility
Srvcheck.exe: Server Share Check
Srvinfo.exe: Remote Server Information
Srvmgr.exe: Server Manager
Ssdformat.exe: System State Data Formatter
Subinacl.exe
Tail.exe
Tcmon.exe: Traffic Control Monitor
Timeit.exe (documented in Readme.htm)
Timezone.exe: Daylight Saving Time Update Utility
Tsctst.exe: Terminal Server Client License Dump Tool
Tsscalling.exe: Terminal Services Scalability Planning Tools
Uddicatschemeeditor.exe: UDDI Services Categorization Scheme Editor
Uddiconfig.exe: UDDI Services Command-line Configuration Utility
Uddidataexport.exe: UDDI Data Export Wizard
Usrmgr.exe: User Manager for Domains
Vadump.exe: Virtual Address Dump
Vfi.exe: Visual File Information
Volperf.exe: Shadow Copy Performance Counters
Volrest.exe: Shadow Copies for Shared Folders Restore Tool
Vrfydsk.exe: Verify Disk
Winexit.scr: Windows Exit Screen Saver
Winhttpcertcfg.exe: WinHTTP Certificate Configuration Tool
Winhttptracecfg.exe: WinHTTP Tracing Facility Configuration Tool
Winpolicies.exe: Policy Spy
Wins.dll: WINS Replication Network Monitor Parser
Wlbs_hb.dll & Wlbs_rc.dll: Windows Load Balancing Server Network Monitor Parsers

Now that we are looking at Windows 2003, I am reminded that i have previously gotten help (RMTSHARE.EXE) from Windows NT resource kit tools! So here is that list. Some of the Windows NT resource kit tools can be downloaded from Microsoft.

ADDUSERS.EXE: AddUsers - Command-line utility, creates or writes user accounts to a comma delimited file.
(Updated) ANIEDIT.EXE: Animated Cursor Creator - Windows-based tool for drawing and editing animated cursors.
APIMON.EXE: API Monitor
ASSOCIATE.EXE
(Updated) ATANALYZR.EXE: AppleTalk network device ANaLYZeR
AUDITCAT.HLP: Audit Categories Help
(New) AUDITPOL.EXE: AuditPol
AUTOEXNT.EXE: AutoExNT Service - Enables you to start a batch file, AUTOEXNT.BAT, at boot time without having to log on to the computer on which it will run.
(Updated) AUTOLOG.EXE: Windows NT Auto Logon Setter

BREAKFTM.EXE: Automated Mirror Break/Restore Utility
BROWMON.EXE: Browser Monitor - Windows-based tool, shows browser status.
BROWSTAT.EXE: Browser Status - Command-line utility, diagnoses browser problems and shows browser status.

C2CONFIG.EXE: Windows NT C2 Configuration Manager
CHOICE.EXE: Input from Batch Files - (MS-DOS 6.0 utility).
(Updated) CLIP.EXE: Clip
(New) CLIPSTOR.EXE
CMDHERE.EXE: Command Prompt Here
COMPREG.EXE - A Win32 character-based/command-line "Registry DIFF" that enables you to compare any two local and/or remote Registry keys in both Windows NT and Windows 95.
COMPRESS.EXE: File Compress - Command-line utility, compresses files. Needed for Setup customization.
(Updated) COUNTERS.HLP : Windows NT Performance Counters Help
Crystal Reports Event Log Viewer - Provides an easy way to extract, view, save, and publish information from the Windows NT system, application, and security event logs in a variety of formats.

dbWeb
(New) DEFPTR.EXE: Default Printer
DELPROF.EXE: User Profile Deletion Utility
DELSRV.EXE
(New) DEPENDS.EXE: Dependency Walker
Desktop Themes for Windows NT 4.0
DESKTOPS.EXE: DeskTops
DFLYDIST.EXE: Compound File Layout User Tool
(Updated) DH.EXE - Command-line utility, enables you to lock heaps, tags, stacks, and objects.
DHCPCMD.EXE: DHCP Administrator's Tool - Command-line utility.
(Updated) DHCPLOC.EXE: DHCP Server Locator Utility - Command-line utility, detects unauthorized DHCP servers on a subnet.
(Updated) DIRUSE.EXE: Directory Disk Usage - Command-line utility, shows disk space used per directory.
DISKMAP.EXE
DISKSAVE.EXE - Enables you to save the Master Boot Record and Boot Sector as binary image files.
DISKUSE.EXE - Command-line utility, scans directories on a hard disk and reports on space used by each user.
(New) DNSCMD.EXE
DOMMON.EXE: Domain Monitor - Windows-based tool, gives status on domains, domain controllers, trust relationships.
DRIVERS.EXE: Device Driver Information - Command-line utility, shows what drivers have loaded.
DSKPROBE.EXE: DiskProbe
DUMPEL.EXE: Dump Event Log - Command-line utility, dumps the event log to a file.

EM2MS.EXE
EMWAC Server CGI Gateway Scripts
ENUMPRN.EXE
EXCTRLST.EXE: Extensible Performance Counter List
EXETYPE.EXE: Finding the Executable Type - Command-line utility, identifies the hardware platform of a .EXE file.
EXPNDW32.EXE: File Expansion Utility - File Expansion utility, expands the compressed files on Windows NT distribution media.

FILEVER.EXE: FileVer - Command-line utility, examines the version resource structure of a file or a directory of files and displays information on the versions of executable files.
(New) FILEWISE.EXE
FINDGRP.EXE: Find Group - Command-line utility, finds all group memberships of a specified user.
(Updated) FIXACLS.EXE: Reset System File Permissions
FLOPLOCK.EXE: Lock Floppy Disk Drives - Command-line utility or service that restricts access to floppy drives.
FORFILES.EXE
FREEDISK.EXE
FTEDIT.EXE: FT Registry Information Editor - Windows-based tool, enables you to create, edit, and delete fault tolerance sets for disk drives and partitions of local and remote computers.

GETMAC.EXE
GETSID.EXE
GFLAGS.EXE
(Updated) GLOBAL.EXE
GRPCPY.EXE: Group Copy

HCLNT4.HLP: Hardware Compatibility List - HCL in online Help format
(Updated) HEAPMON.EXE

IFMEMBER.EXE - Command-line utility, checks whether the current user is a member of a specified group
. IMAGEDIT.EXE: Image Editor - Windows-based tool, enables the creation of icons and cursors, and also used by the Animated Cursor Creator.
Index Server
INSTALLD.CMD (NTDETECT.COM): Startup Hardware Detector
INSTSRV.EXE: Service Installer - Installs any service.

KERNPROF.EXE: Kernel Profiler
KILL.EXE: Task Killing Utility - Command-line utility, use to end one or more tasks, or processes.
KIX32.EXE: KiXtart 95
(New) KIXGRP.EXE

LAYOUT.DLL
LEAKYAPP.EXE: LeakyApp
LINKCK.EXE: Link Checker
(Updated) LOCAL.EXE
LOGEVENT.EXE: Event Logging Utility
(New) LOGOFF.EXE
LOGTIME.EXE

MIBCC.EXE: SNMP MIB compiler
MONITOR.EXE: Performance Data Logging Service and Configuration Tool
(Updated) MUNGE.EXE

NETCLIP.EXE: Remote Clipboard Viewer
NETCONS.EXE: Net Connections
(New) NETDOM.EXE
NETSVC.EXE: Command-line Service Controller - Command-line utility, remotely starts, stops, and queries the status of services.
(Updated) NetTime for Macintosh
NETWATCH.EXE: Net Watcher - Windows-based tool, shows who is connected to shared directories.
NLMON.EXE
NLTEST.EXE
NOW.EXE: Now - Displays the current date and time on STDOUT, followed by any command-line arguments you add.
(Updated) NTCARD40.HLP: Adapter Help - Describes settings for hardware supported under Windows NT.
NTDETECT.COM (INSTALLD.CMD): Startup Hardware Detector
(Updated) NTEVNTLG.MDB
(Updated) NTIMER.EXE
(Updated) NTMSG.HLP
(New) NTRIGHTS.EXE
NTUUCODE.EXE: 32-Bit UUDecode and UUEncode Utility

OH.EXE
OLEVIEW.EXE: OLE/COM Object Viewer
OS2API.TXT - List of compatible APIs in the OS/2 subsystem.

PASSPROP.EXE
(Updated) PATHMAN.EXE: Pathman
(Updated) PERF2MIB.EXE: Performance Monitor MIB Builder Tool
(Updated) PerfLog: Performance Data Log Service
PERFMTR.EXE: Performance Meter - Text-mode utility, provides performance information.
(Updated) Performance Tools
Perl 5 Scripting Language
PERMCOPY.EXE
PERMS.EXE: File Access Permissions per User - Command-line utility.
PFMON.EXE: Page Fault Monitor
PMON.EXE: Process Resource Monitor - Command-line utility.
POLEDIT.EXE: Windows NT System Policy Editor
POSIX Utilities
Power Toys
PSTAT.EXE: Process and Thread Status - Command-line utility, shows process statistics. Useful for debugging problems.
PULIST.EXE
PVIEWER.EXE: Process Viewer - Windows-based tool, shows the processes running in the system and allows ending processes and boosting priority.

QSLICE.EXE: CPU Usage by Processes - Windows-based tool.
QUICKRES.EXE: Quick Resolution Changer

RASLIST.EXE
RASUSERS.EXE: Enumerating Remote Access Users - Command-line utility.
RCMD.EXE: Remote Command Service - Remotely administers and runs command-line programs, client program. Used with RCMDSVC.EXE.
(New) REG.EXE
REGBACK.EXE: Registry Backup - Command-line utility, backs up Registry hives to files without the use of tape.
REGDMP.EXE
(Updated) REGENTRY.HLP: Windows NT Registry Entries - Online Help file
REGFIND.EXE
Regina REXX Scripting Language
REGINI.EXE: Registry Change by Script - Command-line utility, good for Setup programs.
REGKEY.EXE: Logon and FAT File System Settings - Windows-based tool, sets new Registry settings without actually editing the Registry. (Not on PPC RISC-based computers)
REGREST.EXE: Registry Restoration - Command-line utility, restores Registry hives from files.
Remote Access Manager
(Updated) Remote Console
(Updated) REMOTE.EXE: Remote Command Line - Command-line utility, runs command-line programs on remote computers.
Remote Kill
RIPROUTE.WRI: Routing with Windows NT Server
RMTSHARE.EXE: Remote Share - Command-line utility, sets up or deletes shares remotely and can grant and remove ACLs on those shares.
ROBOCOPY.EXE: Enhanced Network File-Copying Utility - Command-line utility.
RSHSVC.EXE: TCP/IP Remote Shell Service
RSHXMENU.EXE: Security Power Toy
RUNEXT: Run Extension

SC.EXE
SCANREG.EXE - A Win32 character-based/command-line "Registry GREP" that enables you to search for any string in keynames, valuenames, and/or valuedata in local or remote Registries keys in both Windows NT and Windows 95.
SCLIST.EXE
SCOPY.EXE: File Copy with Security - Command-line utility.
SECADD.EXE
SECEDIT.EXE
(Updated) SETEDIT.EXE
SETUPMGR.EXE: Setup Manager - Windows-based tool, enables Windows NT to be installed or upgraded remotely.
SETX.EXE
ShareUI
SHORTCUT.EXE
(Updated) SHOWACLS.EXE
SHOWDISK.EXE
SHOWGRPS.EXE
SHOWMBRS.EXE
SHUTDOWN.EXE and SHUTGUI.EXE: Remote Shutdown - Command-line and GUI utilities, remotely shut down a server.
(New) SIPANEL.EXE: Soft Input Panel
SLEEP.EXE: Batch File Wait - Command-line utility, waits for a specified amount of time. Useful in batch files.
SNMPMON.EXE: SNMP Monitor
SNMPUTIL.EXE: SNMP Browser
SOON.EXE: Near-Future Command Scheduler
SRVANY.EXE: Applications as Services Utility
SRVCHECK.EXE
SRVINFO.EXE
SRVINSTW.EXE: Service Installer Wizard
(New) SRVMON.EXE: Service Monitor
(Updated) SU.EXE - Enables you to start a process running as an arbitrary user.
(Updated) SUBINACL.EXE: SubInAcl
SYSDIFF.EXE

TDISHOW.EXE: TDI Tracing Utility - Command-line utility, traces packets going across the TDI layer.
TELNETD.EXE: Telnet Server Beta
TEXTVIEW.EXE: TextViewer
TIMEOUT.EXE
(Updated) TIMESERV.EXE: Time Synchronizing Service - Command-line utility or service.
TIMETHIS.EXE: TimeThis
TIMEZONE.EXE
TLIST.EXE: Task List Viewer
TLOCMGR.EXE: Telephony Location Manager
TOPDESK.EXE: Multiple Desktops - Windows-based tool.
(Updated) TOTLPROC.EXE
TweakUI
TZEDIT.EXE: Time Zone Editor - Windows-based tool.

UPTOMP.EXE: Uni to Multiprocessor Upgrade Utility
USRSTAT.EXE
USRTOGRP.EXE: Add Users to Groups - Command-line utility, adds users to local or global groups from a user-specified input text file.

VDESK.EXE

(New) WAITFOR.EXE
WCAT: Web Capacity Analysis Tool
Web Administration of Microsoft Windows NT Server
WhoAmI
(Updated) WINAT.EXE: Command Scheduler
WINDIFF.EXE: File and Directory Comparison - Windows-based tool.
WINEXIT.SCR: Windows Exit Screen Saver - Logs the current user off after a specified time has elapsed.
(Updated) WINLOGO.DOC: "Designed for Windows NT and Windows 95" Logo Handbook
(Updated) WinMsdP.EXE - Command-line utility, generates a text file of all the information in WINMSD.
WINSCHK.EXE
WINSCL.EXE
WINSDMP.EXE: WinsDump
WNTIPCFG.EXE: Graphical IPConfig Utility

XCACLS.EXE

The Windows 2000 resource kit tools are equally important(jt.exe), here is a (not complete list). You can download some of the Windows 2000 resource kit tools from Microsoft.

Active Directory Sizer (adsizer.exe)
Application Programming Interface monitor (apimon.exe)
Application Security (appsec.exe)
Cluster Quorum Restore Utility (clusrest.exe)
Counter List (ctrlist.exe)
Cluster Verification Utility (clustsim.exe)
Domain Controller Diagnostic Tool (dcdiag.exe)
Delete File and Reparse Points (delrp.exe)
Delete Server (delsrv.exe)
Display Heap (dh.exe)
DHCP Database Export Import Tool (dhcpexim.exe)
Directory Disk Usage (diruse.exe)
Disk Map (diskmap.exe)
Disk Partition (diskpart.exe)
Disk Manager Diagnostics (dmdiag.exe)
List Loaded Drivers (drivers.exe)
Drive Share (drmapsrv.exe)
Dump Event Log (dumpel.exe)
Dump FSMO Roles (dumpfsmos.cmd)
Registry Size Estimator (dureg.exe)
Encrypting File System Information (efsinfo.exe)
Extensible Performance Counter List (exctrlst.exe)
Extract Cabinet (extract.exe)
FAZAM 2000
GetMAC (getmac.exe)
Get Security ID (getsid.exe)
Group Policy Verification Tool (gpotool.exe)
Group Policy Results (gpresult.exe)
GUID to Object (guid2obj.exe)
Heap Monitor (heapmon.exe)
Hard link display tool (hlscan.exe)
If Member (Ifmember.exe)
IIS Migration Wizard (IISMIGrationWizard_Setup.exe)
Installation Monitor (instaler_setup.exe)
File-In-Use Replace Utility (inuse.exe)
Internet Protocol Security Policies Tool (lpsecpol.exe)
Kerberos Tray (kerbtray.exe)
Kerberos List (klist.exe)
Network Connectivity Tester (netdiag.exe)
Now (now.exe)
NT Detect (ntdetect.com)
Open Handles (oh.exe)
OLE/COM Object Viewer (oleview.exe)
Path Manager (pathman.exe)
File Access Permissions per User (perms.exe)
Page Fault Monitor (pfmon.exe)
Process and Thread Status (pstat.exe)
PuList (pulist.exe)
File Copy (rdpclip.exe)
Relog (relog.exe)
RPC Configuration Tool (rpccfg.exe)
RPC Dump (rpcdump.exe)
RPC Connectivity Verification Tool (rpings.exe)
Manipulate Service Principal Names for Accounts (setspn.exe)
SetX (setx.exe)
Performance Data Block Dump Utility (showperf.exe)
File Replication Service (FRS) Status Viewer (sonar.exe)
Near-Future Command Scheduler (soon.exe)
Automated Installation Tool (sysdiff.exe)
Timethis (timethis.exe)
Trace Dump (tracedmp.exe)
Trace Enable (traceenable.exe)
Trace Log (tracelog.exe)
Terminal Server Capacity Planning Tools (tscpt.exe)
User State Migration Tool (usmt.exe)
Virtual Address Dump (vadump.exe)
Who Am I (whoami.exe)
WinStation Monitor (winsta.exe)
Windows NT IPConfig Utility (wntipcfg.exe)
XCacls (xcacls.exe)

Maybe someone knows of a website that does "Windows alternatives for open source tools", similar to "Open source alternative for Windows (commercial) tools"?

How to job schedule or batch control?

2007-12-28T13:35:00.000+01:00

Any IT system administration has a need for some automation, batch control, job scheduling or whatever you want to call it. Such can be setup with cronjobs, at-jobs or scheduled task setups, most likely on the server where the job/application must run.

MSSQL 2005 maintenance plans have the option of running off one server, but executing on another. Similar option should be present for schtasks on Windows server, but as with MSSQL i have not tried it, I have always executed everything on the local machine where the schedule is setup.

Some issues with this standard job scheduling setup will come up as you go along, say you want to know either of the following:

How the execution went?
What is executing right now?
What was the standard output of a previous run?
How long time did the previous jobs take?
Did a job finish before a certain time or within a certain lenght of runtime?
This job should only run if these first jobs have finished properly.

All this and more seems like valid points in any normal IT administration. Some of the things I quite often want to do is:

Add new onetime only jobs, that need to run just once, eg. execute a script that creates a user, deletes a user, or stop/start a service, etc. etc.
Add new permanent jobs, keeping history of changes in start time etc.
Handle schedules of database servers, such as MSSQL, MySQL and Oracle.

More of what I would like in a batch control/job schedule system:

Must have:

Setup applications with several jobsteps consisting of commandlines
Jobs must be startable at certain times
Keep track of history of executions, including time, returncodes etc.
Jobs and applications must be run based on other application dependencies
Timeouts and alternative actions, eg. alarms(email etc)
Gui for monitoring batch progress
Joboutput (standard output) must be available central

Nice to have:

Load evaluation/weight of nodes, deciding where to send jobs.
Failover execution if worknodes fail certain checks (node health check support)
Eliminate the need for a central control server. Most nice would be all nodes to be aware of every other node, allow failover and pick up new nodes if they come alive again.

When looking into what systems can do this I get caught up in a mix of grid computing, load balancing and job/batch scheduling:

There are 31 pages in this section of this category.

Job scheduler
B
Batch queue
BatchMan
BatchPipes
Batchman
C
CONTROL-M
Command queue
Condor High-Throughput Computing System
Cronacle
G
Grid MP
H
IBM Houston Automated Spooling Program
I
IBM 2780/3780
IBM Tivoli Workload Scheduler
IBM Tivoli
Workload Scheduler LoadLeveler
J
Job Control Language
Job Entry
Subsystem 2/3
L
Load Sharing Facility
M
Maui Cluster Scheduler
Moab Cluster Suite
O
Open Source Job Scheduler
P
PTC
Scheduler
P cont.
Portable Batch System
R
RTDA Network Computer
Remote Job Entry
Retriever Communications
S
S-graph
SAP
Central Process Scheduling
SHARCNET
Sun Grid Engine
U
Unicenter
Autosys Job Management
X
Xgrid

Currently we are using TWS, and a homemade system which can do all we need, plus is extendable. Of course TWS is something we are forced to use, the other system would be just fine.

Of course I will be limited to open source or free systems, so I came up with these few systems I would like to try out:

TORQUE is opensource and support available, that is nice for the enterprise. Torque is available for FreeBSD via ports, even very actively maintained.
Sun Grid Engine is a batch queueing system implementing a superset of the
functionality of the POSIX batch queueing framework. Also in FreeBSD ports.

On a side note, i stumbled upon a cluster admin article for unix with ssh, where cssh is suggested, but with much more in comments on http://www.debian-administration.org/.

Snort - what can you do

2007-12-25T20:00:00.000+01:00

Taosecurity heads up on his 11th Snort report which is a good NSM read for most Snort administrators or just NSM interested IT security technician. Reading his books will also get you the idea of a NSM approach :-)

Some snips:

"How do I make Snort log sessions/flows?" It's inspiring to see such faith in Snort, but such questions indicate a certain amount of tool-fixation.

Snort can operate in two modes: active and passive. Snort can be active either inline or offline:

In an active, inline mode, Snort acts as an intrusion prevention system (IPS)...
In an active, offline mode, Snort acts as a quasi-IPS...
In passive, inline mode, Snort sits physically on the wire and allows all traffic to pass...
... passive, offline mode... watches traffic provided by a network tap or switch SPAN port.... is the most popular...

...The following is a transcript generated from Sguil. The data was collected by a second instance of Snort running in pure Libpcap packet logging mode. The content was built using Tcpflow. The operating system fingerprinting was done by P0f...

... This very short example hints at the real power of Snort. I tend to see Snort as a pointer to activities that require additional inquiry. A Snort alert should be the beginning of an investigation, not the end.

Yes it is the NSM story, I like it of course :-)

Oh by the way, I can only agree with the problem of tool fixation. Tools does not solve problems, although many think so still. It requires much more :-) Related to this problem is mis-usage and security by belief (instead of fact) due to systems being setup and operated by "make install".

Windows scheduled tasks, backup/restore/administer

2007-12-18T14:49:00.000+01:00

Recently I had to make an analysis of scheduled tasks on about 50 servers, mixed Windows 2000 and 2003. Some of the tasks was to be recreated on new Windows 2003 servers. Same project as the shares analysis.

I first turned to schtasks.exe which can be used for query (and create on 2003) , for example:
schtasks.exe /S server /delete /f /tn "calc"
schtasks.exe /S server /CREATE /SC ONSTART /TN "calc" /TR "command" /RU:"domain\user" /RP:pass
schtasks.exe /S server /run /tn "calc"

The query output gives information that can be parsed, eg:
schtasks /query /v /fo table
HostName TaskName Next Run Time Status Last Run Time Last Result Creator Schedule Task To Run Start In Comment Scheduled Task State Scheduled Type Start Time Start Date End Date Days Months Run As User Delete Task If Not Rescheduled Stop Task If Runs X Hours and X Mins Repeat: Every Repeat: Until: Time Repeat: Until: Duration Repeat: Stop If Still Running Idle Time Power Management

server calc Never 16:30:00, 12-12-2007 0 user At 16:30 every Mon, Tue, Wed, Thu, Fri of every week, starting 06-12-2007 C:\WINDOWS\system32\calc.exe calc.exe N/A Disabled Weekly 16:30:00 06-12-2007 N/A MONDAY,TUESDAY,WEDNESDAY,THURSDAY,FRIDAY N/A runasdomain/user Enabled 72:0 Disabled Disabled Disabled Disabled Disabled Disabled

But as schtasks.exe does not work on Windows 2000 I turned to jt.exe from Windows 2000 resource kit:
ftp://ftp.microsoft.com/reskit/win2000/jt.zip
3104f01eb01ce8b482bf895db60d7e8e jt.exe

I looked at some jt.exe examples, and created a parser in perl. The basic usage of jt.exe was pretty much limited to:
joblist from: jtbin /sm \"$myserver\" /se p
credentials: jtbin /sm \"$myserver\" /sac \"$jobname\" /gc

Here are some more examples of create commands, generated from parsing the jt.exe output:
Example of mon-fri 8-18, every minute:
schtasks.exe /create /SC WEEKLY /RI 1 /ST 08:00 /ET 18:00 /D MON,TUE,WED,THU,FRI /TN "task" /TR "cmd" /RU:domain\user
Every morning, mon-fri:
schtasks.exe /CREATE /SC Weekly /D MON,TUE,WED,THU,FRI /ST 07:00:00 /TN "task" /TR "command"

Later i found that i can patch SCHTASKS.EXE for Windows 2000 usage, and i turned out to actually work perfect. But i had already used jt.exe output for parsing, and it did do everything i needed. Here are the checksums of the files i tested patching with:

4D918C96C3306DF5F460801437BF24FC schtasks_w2k_5.1.2600.2180_patched.exe 86E33A8D9174DB2DB5001D0FD5DCFB8D schtasks_w2k3_5.1.2600.2180_orig.exe

Some of the problems i have or had while working with scheduled tasks:

Parsing more that the first trigger for a task.

How to make create a task or modify the default task property: "Stop Task If Runs X Hours and X Mins: 72:0". This is a problem if the task is created as ONSTART, but we want it to keep running for ever.

Worked around this by calling a cmd wrapper so the task it self is not running, but a wrapper which loops.

I did not try using "jt /? /sj" option which might be what I needed:

/SJ - set task's properties
Change one or more properties on the in-memory task object.
...
MaxRunTime = (in milliseconds)
...
Example: /sj command = notepad.exe Priority=idle DeleteWhenDone=1

How to make sure a schtask program is started in session 0?

That is, if a terminal service session 1 or 2 exists, the remote schtask /run command will sometimes(not always) start the program in session 1 or 2, which is not always what we want.

Only workaround was to manually logging into terminal services /console and starting task.

So this problem is not solved :-)

If the task is set for ONSTART it will of course start in session 0 if you reboot the server.

If there is a session 1 or 2, it does not work to use psexec eg. like this:

psexec \\server -i 0 -e cmd /C "schtasks.exe /RUN /TN calc"

Psexec -i 0 (default) and -i 2 works fine if it is not a scheduled task that is started:
psexec \\server -d -e calc.exe
psexec \\server -d -i 2 calc.exe

The jt /? /sj does not seem to have a property for what session a scheduled task starts in:

The property list has the form = ...

The task properties and the form of their values:

ApplicationName =
Parameters =
WorkingDirectory =
Comment =
Creator =
Priority = { Idle Normal High Realtime }
MaxRunTime = (in milliseconds)
Idle = (wait & deadline, in minutes)
Interactive = { 1 0 }
DontStartIfOnBatteries = { 1 0 }
KillIfGoingOnBatteries = { 1 0 }
RunOnlyIfLoggedOn = { 1 0 }
SystemRequired = { 1 0 }
DeleteWhenDone = { 1 0 }
Suspend = { 1 0 }
HaltOnError = { 1 0 }
StartOnlyIfIdle = { 1 0 }
KillOnIdleEnd = { 1 0 }
RestartOnIdleResume = { 1 0 }
Hidden = { 1 0 }
TaskFlags = (in decimal)

- must be surrounded by double quotes if it contains spaces
- { m/d/y TODAY }
- any integer

Case is not significant (i.e., IDLE and Idle are both legal).

Verify computers health before allowing network access

2007-12-18T10:17:00.000+01:00

The topic will be interesting to any Windows administrator who worries about what client computers are allowed on the network. I could imagine that many people will have created their own ways of checking, for example before dhcp gives an ip, or blackholing ips if traffic or status of a machine fails checks.

With Network Access Protection (NAP) in Windows Server 2008 there is a new possibility.

Some quotes and hype:

Administrators can enfore policies with NAP, eg. placing clients that fail requirements in quaratine(limited access) or with no access.
Using NAP with DHCP lets you protect all NAP capable clients that get network access from DHCP including Wifi and lan computers.
Windows XP SP3 will include NAP client software. Vista has it by default. Nap client software for XP beta 3 will XP SP2 NAP capable.
NAP is not limited to Microsoft, the system just has to provide the NAP server with its health state. Example: missing!

To use NAP for DHCP you must perform these tasks: (Remember these are just some snips from Windows IT Pro november 2007 issue).

Prepare environment: must have AD with one or more 2003(or 2008) DCs. Must have DHCP on a 2008 machine, eg. a member server in the domain. Open server manager and add Network Policy Server(NPS) which replaces 2003s Internet Authentication Server(IAS). etc etc
Configure health policies: in the NPS console, configure the System Health Validator (SHV) to the client requirements you have. Configure the Health Policy options, select new and check the SHV's you want to use and if they must eg. pass all SHV checks to be considered healthy, eg. automatic update on, hotfixes installed, firewall on, etc etc. Also create a new health policy for clients to be considered non-compliant/unhealthy. etc etc
Create network policies for NAP: in the NPS console setup Network Policies to specify what network access that will apply to eg. unhealthy clients. etc etc
Configure DHCP for NAP: configure one group of scope options for compliant NAP clients and one scope for incompliant clients. Go to properties of the scope in the DHCP console, enable for this scope in the Network Access Protection tab.
Enforce NAP on the client side: use the NAP client console, group policies or netsh (which has new NAP context). You can edit GPOs from Vista or 2008 Group Policy Management Console (GPMC). Start the Network Access Protection Agent service, and automatic of course. On Vista there is a mmc, napclcfg.msc. Netsh command is: netsh nap client set enforcement ID = 79617.
Run a NAP test and check how you can notice if some clients fail. You will probably get a call from the client owner who can not get online.

Btw, Windows 2003 SP1 already had some Network Access Quarantine (NAQ) that helps administrators limit of deny connections to computers that dont comply with a companys security policies. However there are some problems with NAQ:

Only works with VPN, leaving wifi and normal lan connections out of the game!
NAQ is based on scripts that run on the client, which can be hard to create for every firewall or antivirus software you want to check
After NAQ check is completed, the user can disable firewall or antivirus, it will not be detected, and level of access remains the same.

Of course NAP replaces NAQ:

NAP is essentially the replacement for Network Access Quarantine Control and the long-term solution for customers. Microsoft anticipates that partners will provide services and solutions to assist customers with the maintenance of their existing investment or the update of their networks for NAP.
For a detailed comparison of NAP with Network Access Quarantine Control in Windows Server 2003, see Network Access Protection Platform Overview.

So NAP seems like another tool in the box of Windows network administration, just like WSUS is.

When Vista?

2007-12-16T20:02:00.000+01:00

At my work there is a rumor we will switch to Vista by the end of 2008. It might seem far away, but actually I think it is too soon!

I have tried Vista at home, but I skipped it for my XP again! And that was a machine i used for entertaining, multimedia and such. So I really fear being forced to use Vista for getting work done!! It will happen of course, but I hope it wont be soon!!

For a good laugh, read the Upgrade to Windows XP :-) Also on /.

Flickr statistics and Picnik editing

2007-12-16T19:31:00.000+01:00

Finally Flickr added stats for pro accounts, thank you :) Flickr statistics was missing so its nice to see it in action! I would like to be able to go back in time, maybe that will come, so far its a good start!

And with Picnik picture editing in place there is no chance I am leaving Flickr anytime soon! Of course I prefer Google services for most anything else ... youtube, calendar, e-mail, documents and blogging of course :-)

Encrypted filesystems solutions

2007-12-15T11:47:00.000+01:00

I recommed reading the monthly CRYPTO-GRAM, it always has interesting stories from real life security, and not just IT related. Well worth subscribing to. It is often long, but a very good security round up of the month!

This month CRYPTO-GRAM had some nice reflections on disk encryption. Still relevant even after so many years of one story after another where personal data is lost, this latest is no exception!

So it should be no surprise that many people and companies still dont use disk encryption in some form or the other, but it is sad.

Some quotes:

Computer security is hard. Software, computer and network security are all ongoing battles between attacker and defender. And in many cases the attacker has an inherent advantage: He only has to find one network flaw, while the defender has to find and fix every flaw.
...
There are several whole-disk encryption products on the market. I use PGP Disk's Whole Disk Encryption tool for two reasons. It's easy, and I trust both the company and the developers to write it securely. (Disclosure: I'm also on PGP Corp.'s Technical Advisory Board.)

Setup only takes a few minutes. After that, the program runs in the background. Everything works like before, and the performance degradation is negligible. Just make sure you choose a secure password -- PGP's encouragement of passphrases makes this much easier -- and you're secure against leaving your laptop in the airport or having it stolen out of your hotel room.

I am missing whole disk encryption on some of my computers, so i will look into that. On my Macbook i use Filevault.

There are other encryption programs out there. If you're a Windows Vista user, you might consider BitLocker. This program, embedded in the operating system, also encrypts the computer's entire drive. But it only works on the C: drive, so it won't help with external disks or USB tokens. And it can't be used to make encrypted zip files. But it's easy to use, and it's free. And many people like the open-source and free program, TrueCrypt. I know nothing about it.

I prefer TrueCrypt on Windows (didnt work on *nix when i tried a while back), having all sensitive data inside containers. On FreeBSD i use GEOM Based Disk Encryption (gbde) and EncFS.

An interesting twist and point to take note of, is if you are forced to type in your password. By authorities or criminals:

And some countries -- the United Kingdom, Singapore, Malaysia -- have passed laws giving police the authority to demand that you divulge your passwords and encryption keys.
...
Failing that, you can try to convince the authorities that you don't have the encryption key. This works better if it's a zipped archive than the whole disk. You can argue that you're transporting the files for your boss, or that you forgot the key long ago. Make sure the time stamp on the files matches your claim, though.
...
The best defense against data loss is to not have the data in the first place.

You really dont need to walk around with all kind of data, so dont!

IT security, determine your score of the game

2007-12-07T13:36:00.000+01:00

I am not sure why I missed a really good post at Taosecurity, maybe it was the size of the post and me being tired when going over his blog. This post is very important when thinking about IT security, so once again I remind myself to keep reading Taosecurity, even if I am tired :-)

Anyway, some of the key viewpoints, some new to me, some not:

... don't think your security responsibilities end when the bottle is broken against the bow of the ship and it slides into the sea. You've got to keep watching to see if it sinks, if pirates attack, how the lifeboats handle rough seas, and so forth.

And there is an excellent list of suggestion for how to determine your enterprise "score of the game," and use that information to decide what you need to do differently.

Here are some headlines from the list:

Standard client build client-side survival test. Create multiple sacrificial systems with your standard build. Deploy a client-side testing solution on them, like a honeyclient
Standard client build server-side survival test. Create multiple sacrificial systems with your standard build. Deploy them as a honeynet.
Standard client build client-side penetration test. Conduct my recommendation penetration testing activities and time the result.
Standard client build server-side penetration test. Repeat number 3 with a server-side flavor.
Standard server build server-side penetration test. Repeat number 3 against your server build with a server-side flavor.
Deploy low-interactive honeynets and sinkhole routers in your (internal) network. These low-interaction systems provide a means to get some indications of what might be happening inside your network.
Conduct automated, sampled client host integrity assessments. Select a statistically valid subset of your clients and check them using multiple automated tools (malware/rootkit/etc. checkers) for indications of compromise.
Conduct automated, sampled server host integrity assessments. Self-explanatory.
Conduct manual, sampled client host integrity assessments. These are deep-dives of individual systems. You can think of it as an incident response where you have not had indication of an incident yet.
Conduct manual, sampled server host integrity assessments. Self-explanatory.
Conduct automated, sampled network host activity assessments. ... The idea is to let your NSM system see if any of the traffic it sees is out of the ordinary based on algorithms you provide.
Conduct manual, sampled network host activity assessments. This method is more likely to produce results. Here a skilled analyst performs deep individual analysis of traffic on a sample of machines (client and server, separately) to see if any indications of compromise appear.

In all of these cases, trend your measurements over
time...
Don't slip into thinking of inputs. Don't measure how many hosts
are running anti-virus. We want to measure outputs. We are not proposing new
controls.

Key phrases: manual vs. automated and server vs. client, and proactive investigation.

Most of the info has been on his blog before, but all toghether yet another great post :-)

Adminstrating what your DNS queries are: OpenDNS

2007-12-07T13:09:00.000+01:00

It seems like an obvious win for your client network security, when it comes to visiting malicious hostnames: use an DNS server which denies certain hostnames based on some Realtime Block Lists (RBL). Similar to what can be used in parsing e-mails for spam points!

I read several places about OpenDNS, a great free DNS provider who does exactly what you would like, even with added administration to remove blacklists, see top queries etc. And they continue to improve the service and administration dashboard.

So check it out if you are administrating a client network intranet for example. Perhaps its is a bit too far using it for your servers :-)

Starting some PowerShell notes

2007-12-07T12:27:00.000+01:00

For a while there was hype about Microsofts new scripting shell, it was referred to as Monad or MSH, now it is called PowerShell.

A good place to start is at Rob van der Woude's scripting pages:

Getting started:
Download and install Windows PowerShell 1.0 RtW and .NET Framework 2.0 RTM and the Windows PowerShell 1.0 Documentation Pack.
You'll need to uninstall older versions of PowerShell first.
...
PowerShell Links:
Windows PowerShell Quick Start

Here are some notes and tips from Windows IT Pro november issue:

Powershell uses a new set of commands called cmdlets and a new syntax.
Help: Get help with the get-help command.
CD: you can change to registry key: cd hklm:\software
Get-Alias cmdlet is gal, eg. list all aliases: gal select name, definition
Get-Command to see the many commands available, eg: get-command get*
Set-Content to write values to a file: sc c:\f.txt -value "Hi"
Get-Content to read contents of a file: gc c:\f.txt
Set-ExecutionPolicy: by default powershell can not run scripts, you can only enter commands at the command line. To enable run scripts: set-executionpolicy unrestricted
Set-PsDebug: for example step through one line at a time, set-psdebug -step
Get-Process: you can list all running processes: get-process
Get-Eventlog: for example: get-eventlog -newest 10 -logname system

I think I wont get started with Powershell for real until Windows 2008 / Exchange 2007 or similar is being used somewhere close to where i do my administration :-)

Windows shares and NTFS file permissions, show/create/modify

2007-11-30T10:08:00.000+01:00

Recently I had to make an share analysis of about 50 servers, mixed Windows 2000 and 2003. The shares was to be recreated on a new set of servers, including a change for some to Windows cluster server shares.

There turned out to be at least several possibilities:

Export shares, including permissions from registry, and restore on new servers.
Use net share, but it is only good for creating shares, it can not list or modify share permissions.
RMTSHARE.EXE from WinNT ressource kit can do all we need.
Use a VBS script to list and create shares, Win32_LogicalShareSecuritySetting and Win32_ACE.

I went down the VBS script path, and it worked out fine, created a bunch of command oneliners I could use on the new servers or on the new clusters, eg:

cluster . res "share" /priv security="domain\group",grant,F:security
cluster . res "share" /priv security="domain\user",grant,R:security
net share="d:\path\to\share" /GRANT:"domain\user1",READ /GRANT:"domain\user2",FULL

The net share command creates the share, but on the cluster share was created with a wrapper script was made from a Microsoft example, only changing ShareSubDirs=0. Then the above cluster command works fine.

The problem with the script method was that if there was no ACL for a share, my script did not list the share. And i didnt make the script query remote servers, so i used a little psexec workaround in the scripts:
copy listshares.vbs \\server\d$\
psexec \\server -e cmd /C "cscript d:\listshares.vbs"
psexec \\server -e cmd /C del d:\listshares.vbs

Anyway, in the future I recommend using RMTSHARE.EXE which works fine on 2000/2003/xp, can query shares remote, modify permissions, create and all I need. Some examples:

List shares: RMTSHARE \\server
List permissions of a share: RMTSHARE \\server\share /users
Add a user to a share remote: RMTSHARE \\server\share /grant "domain\user":F
Revoke a user permissions: RMTSHARE \\server\share /grant "domain\user"

By the way, note that "net share" command is different on Windows 2003 and on XP. There are permissions options on the Windows 2003 version:
The syntax of this command is:

NET SHARE
sharename
sharename=drive:path [/GRANT:user,[READ CHANGE FULL]]
[/USERS:number /UNLIMITED]
[/REMARK:"text"]
[/CACHE:Manual Documents Programs None ]
sharename [/USERS:number /UNLIMITED]
[/REMARK:"text"]
[/CACHE:Manual Documents Programs None]
{sharename devicename drive:path} /DELETE

There is no permission option on the XP version:
net share /?
The syntax of this command is:

NET SHARE sharename
sharename=drive:path [/USERS:number /UNLIMITED]
[/REMARK:"text"]
[/CACHE:Manual Automatic No ]
sharename [/USERS:number /UNLIMITED]
[/REMARK:"text"]
[/CACHE:Manual Automatic No ]
{sharename devicename drive:path} /DELETE

For NTFS file permissions setting, remove and modify, I use XCACLS.VBS, which can do all we need. It also works on the clusters. Some examples:

Listing access, if you want subdirs add /s /t:
cscript c:\bin\XCACLS.vbs d:\dat\ /server server

Give access, with /e so other users are left as they were:
cscript c:\bin\XCACLS.vbs d:\dat\ /e /g "domain\user":F /server server

Revoke (/r) example, remote: !!! WARNING !!! remember the /e or every permission will be gone:
cscript c:\bin\XCACLS.vbs d:\dat\ /e /r "domain\user" /server server

My only problem with XCACLS.VBS so far, is that it it truncates output of the users, so its hard to wrap into a script for recreation. Eg, it shows only "Domain\Some_domain_gruo" below and not the full groupname:

"Allowed Domain\Some_domain_gruo Modify..."

Ideas for solving this are very welcome :-)

More FreeBSD 7 goodies

2007-11-29T20:00:00.001+01:00

As if the binary upgrade posibilites in FreeBSD 7 (and 6 to 7 if you like) was not enough (and actually working), there are plenty of goodies to look forward to:

SQL database performance ... MySQL 5.0.45 (thread-based)
New filesystems ... ZFS
Network stack changes...Complete elimination of giant lock from network stack
Intel wireless drivers: ... iwi (2200BG/2225BG/2915ABG)...Works out of the box
Atheros protocol extensions...802.11n support (forthcoming standard)...I higher performance: up to 135 Mb/sec
Security subsystems...Audit subsystem... Fine-grained, configurable logging of security-relevant events...System calls, application and user space activities
Performance ... If you find a workload that FreeBSD 7.0 performs poorly on, we want to hear about it!
IPMI (Intelligent Platform Management Interface); monitoring
system hardware

Oh and then some teasers of what to expect in the horizon:

FreeBSD 8.0-CURRENT, due some time in 2009 (maybe)

Some of the features that seem to be lurking on the horizon:
Continued performance optimization, also targetting 16-core
systems (AMD/Intel)
Improved network performance on parallel workloads
Improved filesystem performance
Virtualization support: xen, network stack virtualization, ...
BLUFFS: BSD Logging Updated Fast File System. UFS with
filesystem-level journalling.
Serial Attached SCSI, SATA integrated under CAM (storage
layer also used for SCSI)
DTrace support from Sun; powerful and extensible debugging
and system analysis framework
Stuff we haven't even thought of yet!

I wish i could use FreeBSD for more of my everyday work hehe... :-)

UPDATE: O'Reilly ONLamp had a really great article with loots of details of Whats New In FreeBSD 7.0.

Sysadmin sites to include in your own searchengine crawl

2007-11-29T14:12:00.004+01:00

During the everyday life of a sysadmin Google plays a large role, but also the internal knowledge base is important as there are (should be!) cases related to your specific systems. So hopefully you are providing search for that internal knowledge, or it could go unused!!

I am thinking of collecting a set of external sites to include in our internal search engine crawling, as those sites seems to pop up again and again.

I will build a list of sites to include here, bare in mind this is a raw list, i will update it when they are actually put in the search crawler!

Windows adminstrator/script related so far:
http://www.jsifaq.com/
http://www.windowsitpro.com/topics/index.cfm?action=ArticleList&ChildTopicID=72
http://www.windowsitpro.com/Articles/ArticleID/14459/14459.html?Ad=1
http://www.windowsitpro.com/windowsnt20002003faq/
http://www.ss64.com/nt/
http://www.computerperformance.co.uk/vbscript/
http://www.robvanderwoude.com/
http://cwashington.netreach.net/
http://www.ericphelps.com/batch/index.htm
...
From ss64.com link page there are many *very* good sites, a lot I didnt know before, here some snips:

CommandLine.co.uk - Batch File examples and Utilities
FP Schultze - Batch files
OldNewThing - Raymond Chen's weblog
Heise-security.co.uk - Manage Win XP updates without an internet connection.
Timo Salmi - FAQ's - Useful NT/2000/XP script tricks and tips (tscmd)
Steve Hardy - NT/2K command line scripting
Rick Lively - Commands for every version of Windows and DOS
List of TCP and UDP port numbers
Joeware.net - Admin, AD and Exchange tools.
FP Westlake - Free Win32 console utilities.
Alexander Frink - NT Security Utils, Logoff, Change password.
Bill Stewart - Batch script and Windows admin tools.
Poor Mans SMS - scan a pre defined IP range and list all installed software.
Microsoft App Compatibility - command line tool to collect application info.
Agent Ransack - File Search for Win XP
AnalogX - Screen capture, Terminal Server copy, etc
Autohotkey - Automate keystrokes, mouse clicks.
AutoIT - GUI scripting
Bamboo Software - Scheduled Tasks and other command line utils.
DumpSec, DumpEvt - Dump Event Log, Registry or Security info.
OCS Inventory - Open Source System Management
Filezilla - FTP
Lost NT password
NTFS undelete - undelete files
nu2/Barts Bootable CDs - Admin/Recovery
Trinity Rescue Kit - for recovery and repair of Windows machines
Netikus - Password, Ping, FTP tools.
OptimumX - Utilities by Marty List
UnDelete - Diskeeper

And perhaps:

http://www.microsoft.com/technet/scriptcenter/default.mspx

FreeBSD sysadmin so far:
http://taosecurity.blogspot.com/
...

Windows 3GB limit and applications using > 2GB

2007-11-29T13:36:00.000+01:00

Windows servers with more than 3 GB ram should have a special setting in their boot.ini. This is examples where %systemroot% is c:\winnt\ even on Windows 2003:

[boot loader]
timeout=3
default=multi(0)disk(0)rdisk(0)partition(1)\WINNT
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINNT="Windows Server 2003, Enterprise"
/noexecute=optout /fastdetect /3GB /PAE

[boot loader]
timeout=3
default=multi(0)disk(0)rdisk(0)partition(1)\WINNT
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINNT="Microsoft Windows 2000 Advanced Server" /fastdetect /PAE /3GB

Also after booting with this setting, check your application is actually enabled to use more than 2 GB mem, that is, if you want it to :-) You should be able to enable an application to use more than 2 gb. Microsoft has a nice description:

You can use the Imagecfg.exe file to provide selective use of application memory
tuning in Microsoft Windows 2000. Executable files that can access the 3-GB
address space must have the IMAGE_FILE_LARGE_ADDRESS_AWARE bit set in their image headers. You can set this bit by using the Imagecfg.exe utility; this
utility is included on the Windows 2000 Server Supplement One Resource Kit
CD-ROM . For example, to modify an executable file that is named Test.exe, use
the following command syntax:
Imagecfg -l test.exe

You can check an exe file by running imagecfg test.exe and look for this string:
Image can handle large (>2GB) addresses

For your reference my copy of imagecfg.exe has this info:
5.0.1556.1
835A3281EAC25F18B9A859F68776F167 imagecfg.exe

Of course this will not be a problem when everyone is running 64 bit, which will happen sooner or later. As you might know Windows Server 2008 is the last version to support 32 bit.

Getting 750 GB SATA drives working

2007-11-15T19:33:00.000+01:00

When i got some new 750 Gb drives, I attached them to my standard SATA controller where i had two 300 Gb SATA drives, but then my computer would not start! It did not help to limit the disks by jumper setting:

Moving the drives to the Promise Fasttrak controller got the PC booting, but Windows XP couldnt recognize the drives.

I knew i had to update some BIOS or drivers, but i did not know what motherboard i had. So i turned to a friend, he recommended the freeware CPU-Z tool:
http://www.cpuid.com/cpuz.php

CPU-Z produces all kind of info, you can use the gui, or export to file or html, so run it before you upgrade:

cpuz.exe -txt=%computername%-%date%-before_upgrade

I needed the motherboard info model and current BIOS version:

Mainboard Model MS-6702E (0x1E1 - 0xBE28EE)

DMI BIOS
--------
vendor American Megatrends Inc.
version 080011
date 06/08/2005

I entered the model number into MSI CPU support form and got all kind of nice info about drivers and BIOS. But I also saw the LiveUpdate, which i used instead.

After using driver for the SATA and rebooting, the drives added to the Promise controller was visible to XP :-) And it turned out that my bios was uptodate.

I did not need more support from MSI.

So buttom line is, I got the drives working from the non-standard sata controller :-)

Printkey 2000

2007-11-15T18:27:00.000+01:00

I got a copy of Printkey 2000 5.10 Full from a friend, he has these md5sums:

93C16AF42A3D508F90AED5CCA1DB5D5B PrintKey.exe
DB4BC1B5BF470886D7C495E2E45C8553 Printkey2000.exe

This way i dont have to rely on some download, which I am not sure is safe:

http://www.zdnet.de/downloads/prg/6/y/de000H6Y-wc.html
3033b0d05c7e37999b4b9644f53785af *prntky.zip

SQL queries

2007-11-14T13:53:00.000+01:00

Today I made a view with a simple join, its very easy once you get the hang of it:

select b.column-name-1,a.column-name-2 from table1 a, table2 b where
b.somecolumn = a.somecolumn

Open source alternatives, for MS Project

2007-11-13T10:01:00.000+01:00

I dont think I mentioned osalt.com open source alternatives here before, and today i used it again :-)

It is a great website for your business collegues or management who might not be so familiar with open source and the alternatives available. Please check the osalt sections, and send suggestions to them!

Osalt.com gives you a great overview of commercial software and the alternatives, and even include a list supported operating systems.

Osalt.com does not have everything, as not all great software is open source. My favorite freeware editor pspad is one example, and can not be found on the osalt ultraedit alternative list.

Recently we needed alternatives to MS Project, or at least a .mpp viewer, as the license costs for MS Project is insane. The export to webpage wizard is just not my friend, so a viewer for my collegues is needed!

I would have used openproj, as that works on Mac and Unix and I really just need a viewer, but it requires JRE > 1.5 which I dont have here at work. Besides it was a beta, and if you really need to work with project management, go for Ganttproject which also is available for Mac.

OpenWorkbench which only needs 1.3.1 or later of Sun's Java Runtime Engine, but beware there has not been a release since december 2005.

If you can live with a shareware MS Project viewer, you might checkout Projette. I dont know it will nag or stop working after some days, so far there has been no problems.

Datestring in batch regardless of regional date setting

2007-11-13T09:48:00.000+01:00

A while back I mentioned a collection of advanced batch commands, and today I actually needed the good old env variable %TimeStamp%, so here it is:

@echo off
:: Works on any NT/2K/XP machine independent of regional date settings
FOR /f "tokens=1-4 delims=/-. " %%G IN ('date /t') DO (call :s_fixdate %%G %%H %%I %%J)
for /F "delims=: tokens=1-2" %%i in ('time /t') do (call :settimeenvvars %%i %%j)
goto :s_print_the_date

:s_fixdate
if "%1:~0,1%" GTR "9" shift
FOR /f "skip=1 tokens=2-4 delims=(-)" %%G IN ('echo.^date') DO (
set %%G=%1&set %%H=%2&set %%I=%3)
goto :eof

:settimeenvvars
set hour=%1
set minute=%2
IF 1%hour% LSS 20 SET hour=0%hour%
IF 1%minute% LSS 20 SET minute=0%minute%
goto :eof

:s_print_the_date
set timestamp=%yy%%mm%%dd%
if "%1" == "dateonly" goto :end
set timestamp=%timestamp%-%hour%%minute%

:end
echo %timestamp%

I have mentioned it before, but much inspiration for batch can be found at robvanderwoude.com.

One Windows program I have never had a use for before is c:\windows\system32\attrib.exe, which displays or changes file attributes:

ATTRIB [+R -R] [+A -A ] [+S -S] [+H -H] [drive:][path][filename] [/S [/D]]

+ Sets an attribute.
- Clears an attribute.
R Read-only file attribute.
A Archive file attribute.
S System file attribute.
H Hidden file attribute.
[drive:][path][filename] Specifies a file or files for attrib to process.
/S Processes matching files in the current folder and all subfolders.
/D Processes folders as well.

Example:
attrib file://servername/d$/%2 -r -s -h

FreeBSD binary upgrades

2007-11-12T20:53:00.000+01:00

Finally it looks like there will be a binary upgrade possibility in FreeBSD even for major versions going from 6.x to 7.x. Very cool work, I will definately test it!

An interesting side notes is the recommended method for portupgrade of all ports, it deals with the ruby and ruby18-dbd problems we all know:

Using portupgrade to rebuild everything is a bit tricky since it can get a bit confused when upgrading the programs it uses (ruby and ruby18-bdb), but the following procedure should work:
# portsnap -I update # portupgrade -f ruby ... # rm /var/db/pkg/pkgdb.db # portupgrade -f ruby18-bdb ... # rm /var/db/pkg/pkgdb.db /usr/ports/INDEX-*.db # portupgrade -af

Wuala and cryptographic Snake Oil

2007-11-12T20:45:00.000+01:00

A very interesting post on the free community based online harddisk project Wuala, which also has a pointer to the very good post on 9 signs you might be dealing with cryptographic Snake Oil:

These snake-oil warning signs are neither necessary nor sufficient criteria for separating the good cryptography from the snake oil. Just as there could be insecure products that don't trigger any of these nine warning signs, there could be secure products that look very much like snake oil. But most people don't have the time, patience, or expertise to perform the kind of analysis necessary to make an educated determination. In the absence of a Food-and-Drug-Administration-like body to regulate cryptography, the only thing a reasonable person can do is to use warning signs like these as guides.

All is of course recommended reading :-)

As for Wuala I wouldnt mind giving it a try, I just dont have use for it right now.

Sitemeter stastistics

2007-11-12T08:49:00.000+01:00

When I started writing here I added Google Analytics to the blog, it works really well.

In the past I have been very happy with some simple web statistics like Webalizer and AWStats, so now I have added something similar to those: Sitemeter.

Sitemeter Basic is free, it is simple, plus it can send you stat overview by e-mail. It can also act as a good old counter.

As for added the script code to the HTML on your website it is as simple as for Google Analytics.

Windows media on Mac

2007-11-07T21:25:00.000+01:00

Just found something I need for my Macbook, to view Windows media in Firefox:

http://www.microsoft.com/windows/windowsmedia/player/wmcomponents.mspx
http://www.flip4mac.com/

There was no plugin for Firefox, so i installed it manually.

NSM readup, for later use

2007-11-04T20:47:00.000+01:00

I am still behind my own schedule for my NSM setup, guess my wife and our newborn (2 months old) is taking up most of my time :-)

Anyway, I want to keep a few pointers to good articles and websites for later. Once again from Taosecurity :-)

Russ McRee followed his excellent discussion of NSM and Sguil in the October InfoSecMag with a new article called Argus – Auditing network activity (.pdf), published in the November 2007 ISSA Journal. It's another great read.

UPDATE 1:
Great NSM demo from Taosecurity, using session analysis and full content, basically perfect for education: http://taosecurity.blogspot.com/2007/11/analyzing-protocol-hopping-covert.html

UPDATE2:
Taosecurity again of course: What is NSM? NSM vs. IDS, with pointer to a slide show from 2002 :-) It still holds water! One of the good ones:

“IDS” is only a product; NSM is an operation
incorporating products, people, and processes

Get PCI compliance, and become a better administrator and a stronger team

2007-11-01T11:05:00.000+01:00

In the spring and summer of 2006 I was part of completing a PCI compliance. This was a great a great experience. We achieved and learned so much from the process, and in a very short amount of time, because we had deadline before we was going to be audited. I can only recommend the process to anyone!

Here is a quick rundown of what we used:

Osiris for HIM, on both Windows and FreeBSD. At the time there was no OSSEC.
Central syslog.
Snort with syslog reporting, also to SMS. We played with Sguil as NSM but it was too much network data for the server we had setup. If I was to improve and redo something, this would be it, a server with more CPU and diskspace for.
Improved the FreeBSD (ipfw) and Windows (ipsec) firewall administration by rules being pulled from central CVS server.
Nessus 2.x at the time for penetration testing and remote scanning. Later fully automated and reports sent to Subversion for diff, and to to certain e-mail adresses for completeness.
Webservers, mailservers, dns servers etc got a security check, there was not much to improve.
ClamAV on Windows, antivirus, which does not seem necessary, but it was a demand.
All software/webpages and documentation and scripts (setup/upgrade/changes) goes to CVS for ease of diff and review by the different people responsible of the entire setup.

All in all, it was a great experience for myself, and for the team of people involved. It brought us together in a new way while working toward the goal :-)

I am not the only one who is happy about the learning from being PCI compliant. Here are some snips from his experience, it is very similar to my

I'm using OSSEC (http://www.ossec.net) to monitor the individual
SysLog
files for perceived security issues. OSSEC understands Snort, Cisco PIX,
IPTables, and a host of others.
Additionally, I have OSSEC agents running on each of my servers
(including Windoze), which report back to a central OSSEC Server.

Network Intrusion Detection (Snort):
If you are going to use Snort, I highly recommend that you use the
latest version You'll probably have to compile it from source, but it's
worth it. Snort is sending alerts to my central SysLog server, which
provides a nice and easy central logging repository for Snort alerts.
I'm then using OSSEC to monitor the SysLogs for Snort messages, and
generate alert emails.

Rootkit detection and scanning (RKHunter and CHKRootKit [and OSSEC]):
Never trust a single Rootkit scanner. Both RKHunter and CHKRootKit are
excellent tools, but one could have more/different signatures at
different times.

Network Penetration testing (Nessus 3.x):
I can't stress this enough. If you're going to use Nessus
(http://www.nessus.org), do yourself a favor and install the latest
version.

Layer-7 Firewall (ModSecurity / Apache Proxy):
If you're really serious about CISP, spend the $5000 to purchase a
1-year support contract for ModSecurity (Breach Security
http://www.breach.com). In addition to an immense amount of help with
writing custom rules, you also get a really fast ruleset that's
specifically geared towards PCI Compliance.
One caveat, however, is that you should know a good deal about Perl
Regular expressions if you're going to implement ModSecurity. If this is
an issue for you, you may need to look into other (closed-source,
bleck!) alternatives like F5.
Another Firewall solution that I've been playing around with lately is
Untangle (http://www.untangle.com). Unfortunately, I require ethernet
bonding and 802.1q support, so it's not yet a feasable solution for me
yet. That being said, their Snort front-end can't be beat. And I talked
with a couple of the guys at their Linux World booth recently, who said
that they were going to start bundling Untangle with Ubuntu and other
distros (most of which provide the tools and kernel modules for 802.1q
and bonding).

Per machine firewall (IPTables with Shorewall front-end):
Shorewall is extremely powerful, if not a bit difficult to use. I
wouldn't use it for a gateway machine (although I use it as a
router-firewall between networks on my Corporate network), but it makes
a very good Host-based firewall. The idea here is to only leave the
ports that need to be open, open, and only allow access from the
machines/networks that need access to them. You will need other separate
physical firewalls between you and the rest of the world, as well as
between your servers and your database servers, but you can limit who
and what has access to a specific machine.
Secure Central Backups and Archving (Bacula):
I really love Bacula. It's a bit of a learning curve, but it's GPL'ed,
and it runs on multiple platforms. The features of Bacula rival
NetBackup and Legato, although the interface can be cumbersome to use.
The most important feature is Archival encryption. This indemnifies you
against having to report a lost or stolen tape to all of your customers
(which you shouldn't need to worry too much about if you have a good
backup policy).
Of course, you need to have a solid policy for handling tapes that your
employees must adhere to, that a PCI/CISP auditor must sign off on.
Don't be too wordy. All they need to know is: that machines are backed
up on a regular basis, that certain backup sets are retained for XXX
days/years, that you have a compliant offsite archival policy.

Also, if you've never gone through CISP/PCI before, be prepared for a
lot of long nights, headaches, etc. Try not to get discouraged. It will
be worth it in the end. I can honestly say that I am a much better
engineer for having gone through the process.

Windows IT Pro free utilities

2007-10-29T13:32:00.000+01:00

My notes from Windows IT Pro September 2007 free utilities:

Open Computer and Software Inventory Next Generation (OCS inventory NG), more than just unix or just Windows servers. I stumbled upon this before, but I never made a note of it until now.

Locate PC will send alerts to an e-mail you specify if IP information changes. Useful for theft, but article author says he gets a few false positives when a laptop drops shortly off WiFi.

SyncBack is the article authors current choise for remote backup, and he mentions he has tried many.

SIW, System Information for Windows, can tell you anything about your system. And supposedly it can recover a lost password. This is worth keeping a note of, probably will come in handy.

Wink screencasting, similiar to screenshot, but with time aspect. I have always been looking for a free alternative to Snag-It, and all I have so far i Printkey 2000 which is oold. Maybe this will help me in right direction!

Windows 2008 notes

2007-10-29T12:09:00.000+01:00

Current notes about Windows 2008 status that I found worth noting:

Terminal Services functionality gets much better:

Terminal Services Gateway (TSG) which lets you connect to a TSG and from there to other services. This takes away one, not all, reason to use G/On.
New kind of RDP over SSL. Newer XP and 2003 RDP clients will be able to use this.
Remote Programs, which can be placed on your desktop, running on a remote server over RDP. Takes away one, not all, reason to use Citrix.

Server Core is the ability to install a Windows 2008 server without a gui, or in fact a very limited gui. This is interesting, especially for a Unix server administrator like myself.

Server Core does not have .NET.
Server Core can not use Windows PowerShell functionality as that is .NET based!
Server Core can not yet be bought at a speciel license, it is an option.
Server Core can run IIS, but not with .NET.
Server Core can run DHCP, DNS, WINS, file and printer server.
Server Core can not run Exchange 2007 or SQL server 2005.
Server Core is managed from the command window, which means command line.
Server Core can be GUI adminstrated with MMC from a full blown Windows 2008.

64 bit is here, get used to it! 64 bit considered default for Windows server 2008 installation!

64 bit makes real use of RAM above 4 GB, the space can now be 16 TB.
Windows 2008 will be the last Windows server edition to support 32 bit hardware!

Active Directory now called Active Directory Domain Services (ADDS), introduces some new features.

Read Only Domain Controllers (RODC) and read/write Domain Controllers, insted of all domain contollers of since Windows 2000 and 2003 was read/write.
Active Directory snapshot, which you can load, browse and compare. No need to install a seperate Domain Controller.
Fine grained password policies for people inside same domain. Try run adsiedit.msc.

In Windows 2008 version of Group Policy Management Console (GPMC) has a Find command for searching the GPOs :-)

FreeBSD 7 and release documentation readup

2007-10-25T21:45:00.000+02:00

FreeBSD 7.0 is on its way and as always I like to read about it a good while before I start testing and doing actual upgrades. Usually I read the release notes, but before that I look at schedules and todo lists on FreeBSD website, and now I can add another website to the list of information:

The FreeBSD Release Documentation Snapshot Page is a great starting point for any "readup", especially while preparing upgrades, or generally info on where FreeBSD releases are going. Also useful if you have some hardware you dont know if works with FreeBSD.

Are you secure? Prove it!

2007-10-25T20:58:00.000+02:00

I focus on security and maintainability of the IT services I am involved with. I see many people that do not spend the necessary amount of time on IT service quality assurance of the services they provide, which then raises problems with security and maintainability. Without proper understanding of the services, the IT administration job becomes harder!

Another great blog post on Taosecurity pins much of what I believe in and work from in my everyday job:

Are you secure? Prove it. ... You could expand my five word question into are you operating a process that maintains an acceptable level of perceived risk?

I believe if you can answer yes in the right way, you can often get the bonus of in depth understanding of your IT service, making maintainability a much lesser problem. So the investment into being secure becomes much more than just that!

There was a very interesting reply to the post, one mentioning OSSEC, my current favorite must have system for IT adminstrator, regardless it being Windows, FreeBSD or Linux:

How would you go about performing #7 without some type of SEM? Ideally, you would combine SEM with NSM, which is what I plan on doing. Any suggestions? I've read through several of your posts regarding CS-MARS, etc. and I can understand how SEMs don't give you enough information to act upon alerts as they are alert-centric and usually don't provide you with session data or full content data, but at least they can point you in the right direction of further investigation. They provide you with what Daniel from the OSSEC project calls a LIDS (log-based intrusion detection system) and then do the job of correlating them from numerous devices. So how would you do the above (#7) without some sort of SEM?

SEM = Security Event Management. HTH

My answer to the above would of course be to combine SEM and NMS. I would not rely only on one system, I am using a combination of the following: NSM/IDS and HIM/LIDS/SEM.

Here is the complete list from the post, it is just awesome reading. And with some positive talking for selling the NSM idea, which I am all for!

So, are you secure? Prove it.

Yes. Then, crickets (i.e., silence for you non-imaginative folks.) This is completely unacceptable. The failure to provide any kind of proof is security by belief. We want security by fact.

Yes, we have product X, Y, Z, etc. deployed. This is better, but it's another expression of belief and not fact. The only fact here is that technologies can be abused, subverted, and broken. Technologies can be simultaneously effective against one attack model and completely worthless against another.

Yes, we are compliant with regulation X. Regulatory compliance is usually a check-box paperwork exercise whose controls lag attack models of the day by one to five years, if not more. A compliant enterprise is like feeling an ocean liner is secure because it left dry dock with life boats and jackets. If regulatory compliance is more than a paperwork self-survey, we approach the realm of real of evidence. However, I have not seen any compliance assessments which measure anything of operational relevance.

Yes, we have logs indicating we prevented attacks X, Y, and Z. This is getting close to the right answer, but it's still inadequate. For the first time we have some real evidence (logs) but these will probably not provide the whole picture. Sure, logs indicate what was stopped, but what about activities that were allowed? Were they all normal, or were some malicious but unrecognized by the preventative mechanism?

Yes, we do not have any indications that our systems are acting outside their expected usage patterns. Some would call this rationale the definition of security. Whether or not this answer is acceptable depends on the nature of the indications. If you have no indications because you are not monitoring anything, then this excuse is hollow. If you have no indications and you comprehensively track the state of an asset, then we are making real progress. That leads to the penultimate answer, which is very close to ideal.

Yes, we do not have any indications that our systems are acting outside their expected usage patterns, and we thoroughly collect, analyze, and escalate a variety of network-, host-, and memory-based evidence for signs of violations. This is really close to the correct answer. The absence of indications of intrusion is only significant if you have some assurance that you've properly instrumented and understood the asset. You must have trustworthy monitoring systems in order to trust that an asset is "secure." If this is really close, why isn't it correct?

Yes, we do not have any indications that our systems are acting outside their expected usage patterns, and we thoroughly collect, analyze, and escalate a variety of network-, host-, and memory-based evidence for signs of violations. We regularly test our detection and response people, processes, and tools against external adversary simulations that match or exceed the capabilities and intentions of the parties attacking our enterprise (e.g., the threat). Here you see the reason why number 6 was insufficient. If you assumed that number 6 was ok, you forgot to ensure that your operations were up to the task of detecting and responding to intrusions. Periodically you must benchmark your perceived effectiveness against a neutral third party in an operational exercise (a "red team" event). A final assumption inherent in all seven answers is that you know the assets you are trying to secure, which is no mean feat.

Incidentally, this post explains why deploying a so-called IPS does nothing for ensuring "security." Of course, you can demonstrate that it blocked attacks X, Y, and Z. But, how can you be sure it didn't miss something?

If you want to spend the least amount of money to take the biggest step towards Magnificent Number 7, you should implement Network Security Monitoring.

Advanced Batch File Techniques

2007-10-18T09:20:00.001+02:00

Many Windows administrators keep using batch for small scripts and utils. It is a good idea in the right use!

Batch is good to keep things simple, and the majority of your collegues most likely will feel safe when its in batch, compared to perl, vbs, php, sh or even c/c++ utils. Of course there are people who are so good at programming that they can read code, but most likely they are not your avarage Windows administrator collegue.

I found an advanced batch example, which uses functions in batch, and also writes to files, reads from files and cleanup of course. Its good for some batch education. Also check out the improved version, which does not use files. Impressive :-)

For my own little usage today, i wanted to get the drive from a string, so i can switch the that drive and directory, and at the same time replace / to \ so mkdir and cd works. Looking at the all round really cool scripting pages from Rob van der Woude, and his awesome batch examples, I came up with the test script which does the job:

@echo off
SET STRING=c:/some/dir
ECHO Original string: %STRING%
SET STRING=%STRING:/=\%
SET FIXEDSTRING=%STRING%
ECHO Fixed string: %FIXEDSTRING%

FOR /F "tokens=1 delims=\ " %%A IN ('echo %string%') DO SET drive=%%A
ECHO we are on %drive%

SET STRING=%STRING:~0,1%
ECHO or another way, we are on %drive%

SET STRING=%FIXEDSTRING:~2,9999%
ECHO Dir string: %STRING%

What remains would be how to suck in a config file, or a file with lines of strings that I want to manipulate (eg. a list of filenames). How can I do this? :-) Some places to look could be:

http://home.pcisys.net/~sungstad/useful/PCbatch.html

http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/for.mspx?mfr=true

http://www.maem.umr.edu/batch/stack1.html

UPDATE 1:

My first example broke if STRING had quotations, so instead I came up with this to get drive and dirname, and i also stripped quotes:

REM Find the drive of %localroot%:
FOR /F "tokens=1 delims=\ " %%A IN ('echo %localroot%') DO SET localrootdrive=%%~dA

REM Remove surrounding quotation marks:
FOR /F "delims=" %%A IN ('echo %localroot%') DO SET localroot=%%~A
FOR /F "delims=" %%A IN ('echo %vsspath%') DO SET vsspath=%%~A

REM Now get directory:
set localrootdir=%localroot:~2%

REM Strip any leading /
IF "%vsspath:~0,1%"=="/" SET vsspath=%vsspath:~1%
REM Replace / to \ in VSS path:
set vsspathwin=%vsspath:/=\%

REM Create a tmpfile we can use (this is from MKSNT, not Windows):
FOR /F "usebackq" %%A IN (`tempfile`) DO set envtmpfile=%%A
set envtmpfile=%envtmpfile:/=\%
if exist %envtmpfile% del %envtmpfile%

REM Exit if there was an error earlier, eg. like:
IF ERRORLEVEL 1 set error=true
if "%error%"=="true" exit /b 1
exit /b 0

Some for loop notes:
for /l %a in (start increment final) do
for /l %i in (4 2 10) do echo %i
To work with all lines from a .txtfile or output from a command use:
for /f %i in (c:\file1.txt c:\file2.txt) do echo %i
for /f %i in ('dir /ad /b \\server\share\files*') do echo %i

Remember to use %%i in scripts, and not just %i.

On a side note, do remember that environment variables can be usefull in your batch scripts. Set var=something and set var= to unset. And access them from inside your scripts as you would normal variables:
if defined SOME_ENV_VAR goto somewhere
start some.exe

UPDATE 2:

Turns out running call /? gives you much of the cool information i have missed during the years:

call /?
Calls one batch program from another.

CALL [drive:][path]filename [batch-parameters]

batch-parameters Specifies any command-line information required by the
batch program.

If Command Extensions are enabled CALL changes as follows:

CALL command now accepts labels as the target of the CALL. The syntax
is:

CALL :label arguments

A new batch file context is created with the specified arguments and
control is passed to the statement after the label specified. You must
"exit" twice by reaching the end of the batch script file twice. The
first time you read the end, control will return to just after the CALL
statement. The second time will exit the batch script. Type GOTO /?
for a description of the GOTO :EOF extension that will allow you to
"return" from a batch script.

In addition, expansion of batch script argument references (%0, %1,
etc.) have been changed as follows:

%* in a batch script refers to all the arguments (e.g. %1 %2 %3
%4 %5 ...)

Substitution of batch parameters (%n) has been enhanced. You can
now use the following optional syntax:

%~1 - expands %1 removing any surrounding quotes (")
%~f1 - expands %1 to a fully qualified path name
%~d1 - expands %1 to a drive letter only
%~p1 - expands %1 to a path only
%~n1 - expands %1 to a file name only
%~x1 - expands %1 to a file extension only
%~s1 - expanded path contains short names only
%~a1 - expands %1 to file attributes
%~t1 - expands %1 to date/time of file
%~z1 - expands %1 to size of file
%~$PATH:1 - searches the directories listed in the PATH
environment variable and expands %1 to the fully
qualified name of the first one found. If the
environment variable name is not defined or the
file is not found by the search, then this
modifier expands to the empty string

The modifiers can be combined to get compound results:

%~dp1 - expands %1 to a drive letter and path only
%~nx1 - expands %1 to a file name and extension only
%~dp$PATH:1 - searches the directories listed in the PATH
environment variable for %1 and expands to the
drive letter and path of the first one found.
%~ftza1 - expands %1 to a DIR like output line

In the above examples %1 and PATH can be replaced by other
valid values. The %~ syntax is terminated by a valid argument
number. The %~ modifiers may not be used with %*

A good explanation of the FOR LOOP possibilites can be found here [http://www.computerhope.com/forhlp.htm]:

eol=c
specifies an end of line comment character (just one)
skip=n
specifies the number of lines to skip at the beginning of the
file.
delims=xxx
specifies a delimiter set. This replaces the default delimiter
set of space and tab.
tokens=x,y,m-n
specifies which tokens from each line are to be passed to
the for body for each iteration. This will cause additional variable names to be
allocated. The m-n form is a range, specifying the mth through the nth tokens.
Ifthe last character in the tokens= string is an asterisk, then an additional
variable is allocated and receives the remaining text on the line after the last
token parsed.
usebackq
specifies that the new semantics are in force, where a back
quoted string is executed as a command and a single quoted string is a literal
string command and allows the use of double quotes to quote file names in
filenameset.

PC Decrapifier, for Windows

2007-10-17T21:00:00.001+02:00

Before I started this blog I read ten free security tools. Just a few days ago, I wanted to use the decrapifier that was mentioned, but I had forgot the name. So now its here for future ease of use:

The PC Decrapifier does exactly that -- removes crapware that comes pre-installed on Windows computers.
This program will not remove crapware from older computers but is perfect for new machines that ships with trialware.
There is a long list of products it will find and remove, including QuickBooks Trial, NetZero Installers, Earthlink Setup Files, Google Desktop and the myriad of anti-virus trialware apps.

Others from the list worth mentioning:

File Shredder is a must-have privacy tool that wipes/destroys documents beyond recovery.

GMER, a free rootkit scanning tool built by Polish Windows internals guru, is widely hailed as the best at ferreting out stealth rootkits from PCs.

America Online's Active Virus Shield, powered by Kaspersky Lab, is one of the better free anti-virus packages available for Windows users.

OpenDNS is a must-have free service (there's no software to install) that speeds up Web surfing, corrects domain typos on the fly and protects you from phishing scams.
All you do is change your DNS settings (instructions here) to the OpenDNS servers: 208.67.222.222 and 208.67.220.220

Trying OmniFind Yahoo Search

2007-10-17T09:30:00.000+02:00

When looking for Windows search util solutions, I stumbled upon OmniFind, which seemed too good to be true:
Install it in 3 clicks, configure it in minutes.
Free, searches up to 500,000 documents.
Search both the enterprise and the Internet from a single interface.
Incorporates open source Apache Lucene technology to deliver the best of community innovation with IBM's enterprise features.
But OmniFind was exactly like that! Downloading, installing, configuring, testing indexing a website and a filesystem location, all done in 15 minutes!

The server OS requirements are not my favorite, but for the enterprise it makes sense, and expected when it comes to IBM. Their favorites are of course Redhat and Suse. Too bad for me, my favorite Linux being Debian, and of course i always vouch for FreeBSD.

32-bit Red Hat Enterprise LinuxVersion 4, Update 3
32-bit SUSE Linux Enterprise 10
32-bit Windows XP SP2
32-bit Windows 2003 Server SP1

Some notes from the testing so far:

Indexing filesystems, with .doc, .xls, works like a charm, and the search results can be browsed "as html" and "cached". Very useful!

OmniFind installs as its own webservice, on a port of your choice. I changed the search page appearance with company logo and disabled all the Yahoo links. All very simple from the OmniFind admin control panel!

Searching for a string inside any word, you should add a wildcard. For example you should search "regression*" to make sure you locate occurrancies of "regressions".

Reindexing seems to be something you have to wrap into your own scripts, and schedule them, eg. with at jobs.

You can use scripts to start or stop a crawler.
Crawler management scripts allow you to schedule and execute start and stop crawler actions, or start and stop a crawler from the command line.

Cleaning the index for documents that should not be crawled is not so friendly. It seems you have to delete the entire source, eg. website, then crawl it again. It can be tiresome if it is a big website.

The language pack should be installed before you start crawling your big sources, as you will have to do it all over again when then language pack has been installed.

Crawling protected websites was possible, i have tested https:// protected by basic authentication, it worked fine. Crawling formbased authentication, as a company portal document handling system, should also be possible:

HTML form-based authentication
Form name (optional)
Example: loginPage
Form action
Example: http://www.example.org/
authentication/login.do
HTTP method: POST or GET
Example: POST
Form parameters (optional)
Example: userid and myuserID

So far, I am very pleased with OmniFind, I recommend everyone give it a try. OmniFind might be the single point of entry for knowledge search that your organization need to bring knowledge from many sources to life and use!!

Windows date and regional setting

2007-10-16T16:04:00.001+02:00

Different regional settings on Windows servers will cause date command to give different output, which can be annoying if you want to use a date string in your batch scripts.

So I was very happy to see a genious solution:

   @echo off&SETLOCAL

   :: This will return date into environment vars
   :: Works on any NT/2K/XP machine independent of regional date settings
   :: 20 March 2002

   FOR /f "tokens=1-4 delims=/-. " %%G IN ('date /t') DO (call :s_fixdate %%G %%H %%I %%J)
   goto :s_print_the_date
  
   :s_fixdate
   if "%1:~0,1%" GTR "9" shift
   FOR /f "skip=1 tokens=2-4 delims=(-)" %%G IN ('echo.^date') DO (
       set %%G=%1&set %%H=%2&set %%I=%3)
   goto :eof

   :s_print_the_date
   echo Month:[%mm%]  Day:[%dd%]  Year:[%yy%]
   ENDLOCAL&SET mm=%mm%&SET dd=%dd%&SET yy=%yy%

Intranet and file system search tools on Windows

2007-10-16T14:58:00.000+02:00

Recently I have looked into challenges and requirements for search tools for knowledge management. In my testing, I have been focussing on tools that could run off a Unix box, indexing serveral sources of information. Testing those tools are still undergoing.

Now I have another use for search tools, this time running off a Windows server. Requirements for eg. what sources to index are the same as the Unix tools still being tested.

Using the very good searchtools.com website, I found some interesting tools:

Mnogosearch Windows
Zoom search engine
Apache Solr
OnmiFind

So far I have setted up the Mnogosearch for Windows MSSQL with SQL Express 2005, but I still have to setup search integration into IIS. I have stalled this test, mainly because of the price! It is so very expensive, I could almost get a GSA mini instead. For testing the trial version indexing 1 kb of data from each file is okay, but its just too expensive to put more work into. Add to that, it seems that the Windows version is falling behind in releases, does not seem to be maintained very much.

I have not tested Apache Lucene Solr yet. It can become hard to test for me, as it is Java based, and I dont have a ready to run test environment for such testing. Reading on Solr, it should be able to index intranet, hopefully shared drives too, but i have to look at it!

OmniFind, like Solr, is based on Lucene, but seems like a better package for me to test. It is free, can index file system and sounds too good to be true:

Install it in 3 clicks, configure it in minutes.
Searches up to 500,000 documents.
Search both the enterprise and the Internet from a single interface.
Incorporates open source Apache Lucene technology to deliver the best of community innovation with IBM's enterprise features.

I have installed the Zoom search engine on my laptop, indexing the directory with some .doc, .txt, .cmd etc files, putting the result search page to an IIS webserver! Simple and working! In the free version Zoom will only index static files, and a max of 50 documents. This is annoying, I would rather have full version in eg. 30 days! Notes so far:

Cheap, $99 for pro, $299 for enterprise use.
Very easy setup
Search does not trigger documents which have the searched word in filename!
Can reindexing be automated?

Search tools, challenges and non-trivial requirements

2007-10-16T14:10:00.000+02:00

I have listed some key challenges for my current usage of search tools:

Create a point of entry for search.
Link to relevant search query from a portal (eg. a operation status website).
Some knowledge should only be available to some people. This seems to the biggest hurdle!

Limiting knowledge/search only to some people could be solved in at least 2 ways:

Set up different indexer/crawler configurations, each searchable from different search prompt. Problem could be multiple crawls of the same info (load, storage, ressources)
Index/crawl everything once, and let the search box/website/frontend control who can see what. This would be preferred.

Listing non-trivial requirements which are not always availble:

Parse open office word and calc, (.odt and .ods), which is basically zipfiles with xml (unzip and parse eg. content.xml).
Crawling/indexing file sytems (shares/harddrives), setting a baseurl for how the searchresults will become browsable.
Reindexing must automated, eg. scheduled or cron'd.

Windows package management

2007-10-10T21:46:00.000+02:00

I love apt-get that is in Debian, it is my favorite Linux package management system, so I was happy when I stumbled upon Win-get:

win-get is an automated install system and software repository for Microsoft Windows written in pascal (for the command line client) and php for the online repository. The ideas for its creation come from apt-get and other related tools for the *nix platforms.

Recently I am not spending too much time on my Windows client installations, so I probably wont try Win-get, and stick with manually updating.

If I had more Windows clients to maintain, I most certainly would make them logon to a Samba Windows domain and use WPKG for package management.

FreeBSD system beep and dual monitor setup

2007-10-07T10:14:00.000+02:00

I am using PCBSD on my R60 laptop, and turning off system beep was even easier than using kbdcontrol -b off or sysctl.

I simply used the Bell Settings from the KDE menu, and setted volume to 0, simple and effective! The sound system still works like a charm.

I have giving up on setting up dual monitor, with our TV, as everything I found so far points toward a lot of xorg.conf tweaking, something I just dont want to spend my time on!
I have decided that my next laptop must have dual output, eg. a VGA or DVI, and then I just might give it a shot.

I still hope to find a way to change back and forth with the R60 VGA output from laptop to TV. That would be nice to have!

Knoppix-NSM

2007-10-05T20:55:00.000+02:00

I am still playing around with my FreeBSD laptop, trying to tune it for NSM (snort, sguil etc) and penetration testing (nessus 2 and 3), but only moving slowly forward.

So when I stumbled uppon Knoppix-NSM in a NSM and Sguil article, I thought: why spend all this time tuning my laptop if I can boot a liveCD and be running?

Well, I havnt tried the Knoppix-NSM LiveCD, because when I think about it, I enjoy learning while playing and tuning with setting up the FreeBSD laptop for what I need. And I can live with the delay, as I dont have anything I must scan or NSM right at this moment! And perhaps daily work will be ore automatic when I am done, which is also a major concern.

Comparing Office documents

2007-10-05T14:34:00.000+02:00

I found a website with good overview of different diff utils, it has utils i already know, in addition to many others, unfortunately it does not seem like there are any utils which can be customized like i really want.

Even though most of the tools are Windows based and not open source or freeware, I simply have to give a few trials a try, let alone for the following possibilities:

Custom file filters
Command line: supported
Plug-ins for: data files (CSV), image formats, exe/dll version information, mp3 files, icon/cursor files, MS Office and others

Snips from the diffutils.com reviews:

Beyond Compare 2 is a great software solution for almost any revision control project. It has powerful merge and synchronization functionality. However, Beyond Compare 2 is not completely suitable for the comparison of MS Excel and Word files. If your comparison project includes largely MS Office documents, we advise you to use Compare Suite.

Our rating for Compare Suite is 8/10. However, it is our Editor’s choice as one of the best software applications for office document comparison. It also has powerful capabilities for integrating with document management system. In conclusion, we definitely recommend Compare Suite as the optimal choice for document comparison.

Take a look at the full utils list on http://www.diffutils.com/list-of-reviewed-software.

Defining crucial changes for text files

2007-10-05T11:59:00.000+02:00

Sometimes I wish to be able to define what I think is a crucial change for a text file, instead of just every diff from one version to the next which we have plenty of tools for.

I need more than just an average diff util to see what has actually changed from one report output to the next, in order to avoid false positive line matches.

Some of the problems with standard diff util is that it can not handle these cases:

The order of rows has changed, but the content has not changed. (Moved lines detection in file compare)
The offset of columns has changed, but the content has not changed.
Whitespace, tabs and or spaces could be ignored.
Data in a line has changed, but is ok to ignore, such as date changes.
More or less data in a certain section has changed, but can be ignored.
Tags order changes, but the content within a tag does not. Eg. HTML tags.

Some examples of when a more advanced diff util could come in handy is:

Nessus .nsr scan result files, looking for interesting changes.
WYSIWYG HTML editors saves tags in another way that when file was loaded, even if there was no changes.

One approach is to make a configuration file for the diff util so you can use it in as many places as possible. Is this referred to as custom file filters.

Commandline is required for scripted compare.

Unix version is almost a must. Because often it is output from unix boxes that will be compared!

All this, instead of writing a custom parser for diff everytime a new usage comes up :-)

Taskmanager - administrator mode

2007-10-04T12:06:00.000+02:00

It turns out you can get a taskmanager with administrative rights, without being administrator!

What you do is:

Add yourself to administrator group.
Start taskmanager and click [v] Show processes from all users.
Stop taskmanager and remove yourself from administrator group.
Start taskmanager as before, it will now show processes from all users! You can not change the [ ] Show processes from all users, as expected, but it acts like it is [v] :-)

This is useful if you want to get eg. a taskmanager with admin rights on a Citrix server and only have one taskmgr.exe published. Repeat the steps above for each user that needs the show processes from all users!

Supposedly this must be a Windows bug, but I didnt find anything about it.

Cyber Security Awareness Month

2007-10-04T11:31:00.000+02:00

Over at the SANS Internet Storm Senter (ISC), there was an article describing how they will put focus on the october Cyber Security Awareness Month - Daily Topics:

October is Cyber Security Awareness Month and the Internet Storm Center is going to focus on one security awareness subject per day. We plan to provide useful information for information security professionals who want to educate their users but do not have a ready set of awareness tips.

I will keep an eye on the topics, and keep my diary updated with interesting snips as always :-)

Unix utils on your Windows box, eg. quick cleanup of dirs

2007-10-02T11:03:00.000+02:00

With MKS Toolkit or GnuWin32 on a Windows box you can reuse most of your unix oneliners. Very handy for simple administration. There are other unix tools for Windows, but it seems GnuWin32 is very active!

Here is an example, how to cleanup .txt files created more than 35 days ago:

sh -c "find \\\\host\\sharename -name \"*.txt\" -ctime +35 -exec rm \"{}\" \";\""

That is a nice quick way to delete files older than a specific date, it can easily be modified to do more complex stuff.

More complex sample:

sh -c "find \\\\host\\sharename -name \"*.log\" -depth -mtime +0 -exec echo \"{}\" %date% \";\" grep -v \"renamed\" awk '{print \"mv \"$1\" \"$1$2$3\"renamed.log\"}' sh "

EDIT1:

The %date% syntax is really useful in batch scripts, eg:

echo %time%

11:02:54,16

echo %date%

03-10-2007

echo %date:~6%%date:~3,2%%date:~0,2%-%time:~0,2%%time:~3,2%%time:~6,2%

20071003-110300

The end.

Edge based network management and Event tracing for Windows

2007-09-30T22:12:00.000+02:00

Reading another great post from TaoSecurity, this time about Microsofts Anemone project which is an abitious network and systems monitoring system, using network end points.

Anemone is investigating network and systems management from the edges of the network, initially focusing on enterprise network management. It aims to build a network management platform based around two main components: (i) endsystem flow monitoring, providing the inputs to the system; and (ii) monitoring of the network routeing protocols, providing current system configuration. By aggregating and querying these data sources in a distributed fashion, Anemone will provide a platform on which network management applications can be built to provide tools for visualization, what-if analysis, and control of the network.

While edge based approach seems interesting, it is research and out of my leage. I leave it to experts like Richard Bejtlich to give a review at a later time :-) A review I will read with much interest if and when it arrives!

Anyway, the original post mentioned use of event tracing for Windows:
To evaluate the per-endsystem CPU overhead we constructed a prototype flow capture system using the ETW event system [Event Tracing for Windows]. ETW is a low overhead event posting infrastructure built into the Windows OS, and so a straightforward usage where an event is posted per-packet introduces overhead proportional to the number of packets per second processed by an endsystem.
It sounded intesting, going on to the Microsoft website explanation:

Event tracing is a technique for obtaining diagnostic information about running code without the overhead of a checked build or use of a debugger. An event represents any discrete activity that is of interest, especially with respect to performance.
Developers can implement event tracing in a driver by using the Microsoft Windows software trace preprocessor (WPP). WPP software tracing in kernel-mode drivers supplements and enhances Windows Management Instrumentation (WMI) event tracing by adding conventions and mechanisms that simplify tracing the operation of a driver. WPP event tracing is implemented by adding certain C preprocessor directives and WPP macro calls to the driver source code. During an event tracing session, WPP logs real-time binary messages that can subsequently be converted to a human-readable trace of driver operations.

This is interesting, but for the developer, with source code access.

I dont see how Event tracing can help an administrator trace and debug events on servers or clients. Perhaps I am mistaken?

IRC bots and announcements

2007-09-28T20:22:00.000+02:00

I am still in some IRC channels where it would make sense to have announces from work related stuff, eg:

Nagios monitoring alerts.
Subversion commit messages. And other post/pre-commit hooks.

In an ONLamp article about subversion and traq on FreeBSD, there is also an example with a irc announce using RSS feeds, which is implemented with http://supybot.com/. This seems interesting, maybe it is useful for my other needs, I will have to check it out.

On top of the Nagios alert messages, it would be nice to be able to send a query from an IRC bot to the Nagios service to get current state of a service monitor.

Windows 2008 RC 0 and IIS 7 tips

2007-09-26T21:01:00.000+02:00

A few days ago Windows 2008 RC 0 was announced! I will not have time to test it any time soon, but it is a reminder that Windows 2008 will arrive soon, expected already in february 2008!

It will be a nice signal to send to your customers that "We are now testing Windows 2008, IIS7". This will bring momentum to the later post of "Your services is now running on Windows 2008". You will look professional, technical and "process" strong, spending time to prepare and test services early and hopefully thoroughly on Windows 2008! Add to it, that most everyone will agree there was a big improvement going from Windows 2000 to 2003, so your services will benefit from an early, and tested, adoption of Windows 2008!

Reading about Windows 2008, I stumbled at the IIS community website, where there are very interesting articles, for example of how to get Frontpage 2002 running on IIS7, and a pointer to a IIS debugging tool for locating problems with IIS applications crashing etc.

Firefox as a security tool

2007-09-25T20:39:00.000+02:00

I saw this amazing collection of security plugins for Firefox, called FireCAT. I havnt had time to install the collection, but I hope I get a chance soon :-)

Compact server and a laptop for client computer

2007-09-24T22:17:00.000+02:00

Recently I have been preparing my IBM R 60 laptop for network and server administration work, while at the same time keeping it at a functional client level. My conclusion ended at PC-BSD 1.4 a while back, and I have not regrettet that. I can reusage my server automation and administration setup and scripts, and I can use it as a real laptop client computer. Of course using it as a client does violate my own feeling of security, as it has so much installed that I dont use. But it is a good base for my hobby automation and administration projects.

I do feel the combination of serverusage and client on same installation, is a bit opposite and not really good for all future. So I was really pleased to see my favorite blog and book author having an article describing a compact server type computer for his network security monitoring. That setup looks very nice, is AMD based, with lots of disk and expansion options, and he got FreeBSD installed without a problem.

So, when I get the chance to split client usage from server usage, I know what I will get :-) But with the amount of time I have for home server and security projects at the moment, I will stick with my laptop for both server and client computing for a while :-)

Oh, as a bonus Richard reminds his readers of Gconcat, in case that article was missed. Just awesome blogging, I love it :-)

Desktop heap and GDI objects, usage and monitoring

2007-09-19T12:55:00.000+02:00

When working with Lotus Notes and many IExplorer windows you might run into problems with random applications that will not open a new window or Windows will even throw an error:

Initialization of the dynamic library \system32\[kernel32user32.dll] failed. The
process is terminating abnormally.

On my normal laptop there is no problem yet, here is my current physical and free memory:

wmic MEMLOGICAL get TotalPhysicalMemory
TotalPhysicalMemory2087256
wmic OS get FreePhysicalMemoryFree

PhysicalMemory1437220

The taskmgr shows this:

So really, I am not in any problems yet! But if you do have the problem someone wrote about a fix for running out of GDI objects. He describes how it is actually a problem with the desktop heap settings and links to Windows Desktop Heap Tweak Guide and Microsofts own description of the problem.

To sum the solution it should be fixing this registry key:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session
Manager\SubSystems\Windows

My default setting is:

%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows
SharedSection=1024,3072,512 Windows=On SubSystemType=Windows ServerDll=basesrv,1
ServerDll=winsrv:UserServerDllInitialization,3
ServerDll=winsrv:ConServerDllInitialization,2 ProfileControl=Off
MaxRequestThreads=16

Possible change:

Windows SharedSection=1024,3072,512
to:
Windows SharedSection=1024,8192,2048

I am not currently aware of the number of GDI objects where i will run into problems, but i hope to get an example from my collegue.

For the future I am interested in knowing how many GDI objects are created and where in these two cases, using a specific application from Citrix:

GDI object count on client, when application running on client

GDI object count on citrix server and client, when application is started from Citrix WI

I would like some util to monitor and alert on GDI object usage. Some ideas:

Task manager, add the column, this is easy.

Maybe Process Monitor, now maintained by Microsoft, can be used? MISSING.

WMI should be possible, but i have not found the path or alias to use in wmic:MISSING

Add counter in perfmon: MISSING INFO

My other notes during all this reading:

Overview of Performance Monitoring seems to show that GDI objects are not monitored in system monitor. I dont know if this is correct.

This article could be interesting to learn from: How to Use Remote Tools to Track Memory Leaks in Windows CE Applications.

EDIT 1:
I am still looking for tools for GDI object monitoring:
http://www.google.dk/search?hl=da&client=firefox-a&rls=org.mozilla%3Ada%3Aofficial&hs=m2Z&q=wmi+class+for+gdi+objects&btnG=S%C3%B8g&meta=

I am not the only one missing GDI count for a process:
http://www.ureader.com/message/33360788.aspx

There is a monitoring tool, Usage Monitor 1.8.0.3, I tried it, but you can only put a watch limit for one process, not a total limit. But watches can be placed on: Memory Usage, GDI Objects, and USER Objects.

679F88EA6D30D0035E26EC5B88E64063 umon-1.8.0.3.zip

Another monitor tool:
http://www.mmdfactory.com/logger.html

EDIT2:
Another fix was suggested by a collegue:

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Windows]"GDIProcessHandleQuota"=dword:00007530

To check it from batch file, which runs regedit /s file-with-above-content:
Set ExeError=%ErrorLevel%
If .%ExeError%==.0 Set RCValue=%ExeError%

Another way to check GDI object usage was suggested: process explorer.

Avoid make install services, and ideas for best practise IT administration

2007-09-16T20:13:00.001+02:00

In the past I have seen a more or less make install service installation of OpenLDAP as a Samba backend service. Unfortunately there was invested way too little effort on getting a feeling of what the OpenLDAP service was actually doing. The installation was missing basic testing functionally, monitoring, redundancy and missing continuing upgrading. If you do manage to get time or collecting proper knowledge, make sure you store your findings somewhere useful for yourself and your collegues. You might get inspired by my thoughts of knowledge management and single point of entry for search.

I fear there are many IT departments that still perform IT operation as single individuals, not sharing knowledge and sticking with make install service installation. What puzzles me about this picture is how anyone working professionally with IT administration can be satisfied with just make install installations, let alone how can their boss let it happen in their IT department.

From my years of IT administration I have come think of an IT service as something which needs much more than make install! Off my head I can think of at least issues if someone asks me for ideas for a best practise IT adminstration:

service usage understanding(at least basic)
redundancy and availability (high)
security issues, impacts
installation, dependancies
monitoring, logging, baseline for behaviour and files used
performance, baseline and tuning
backup/restore
perform cases of most likely actions, eg. add/remove/change/stop/start
upgrades, minor and major, possibly backup->install/upgrade->restore
locate community wikis, forums and announce mailing list
let someone else setup a complete test environment, following the intial docs
make some (initial) support scripts and docs, which everyone can commit to in the future... knowledge sharing!

All is part of an IT service, and most likely it wont be the same single person performing all aspects for ever, so knowledge sharing is paramount. It may sound like going for the impossible, but I have seen it work out just fine, and to the pleasure of everyone! It makes a great feeling for everyone when everyone can contribute.. it just enforces the good feeling and good work of the department! So keep striving, if you, your boss and your colleguages really want it, you will succeed!

Well anyways, what got me thinking about all this today, was an OpenLDAP post over at OnLamp, which mentions lastest OpenLDAP upgrades, version 3, and a rundown of how to make an OpenLDAP installation redundant. The last part was particular interesting as it mentions syncrepl as superior to slurpd, since OpenLDAP version 2.2:

In the late 1990s, a new feature called Content Synchronization (see RFC 4533) offered a new basis for replication. In OpenLDAP 2.2, the project introduced synchronization replication (syncrepl) based on persistent search. syncrepl uses change sequence numbering and is a pull approach by the replica server. It is much more robust replication approach and more forgiving when replica servers lose connectivity.

I have seen problems with citrix access gateway (CAG) logon failures due to a missing OpenLDAP upgrade, and I have also seen non-working OpenLDAP slurpd replication.

All together, it confirms me in my intial point: avoid make install service installations, and spend more time with your IT service, it will most likely come back in terms of better operation, service and support from your service!

Storm growth and botnets in general

2007-09-13T21:02:00.000+02:00

I am not much involved (recent job changes) with highly exposed servers and services (webhosting) anymore so my time spent eg. fighting spam (botnets play an huge role here of course) has decreased.

After the recent job changes, my interest in security has not decreased, but focus seems to have moved, more toward intrusion/extrustion detection and penetration testing. I hope I can keep exploring that path, with some interesting posts here.

But anyway, after reading about the continuing Storm botnet growth, I wanted to take some of my old notes and urls to this blog. Here is a quick list of good urls to get you started with botnets in general:

And some posts about the Storm botnet structure, growth and operation:

Google analytics

2007-09-13T11:43:00.001+02:00

Yesterday I mentioned a wish for statistics on this blog, and I came to think of Google Analytics (GA). I have never used GA before, but when I was involved in website hosting I came across Urchin, because I was testing the Urchin FreeBSD port. Later Urchin was acquired by Google and is now GA. Urchin was superior to any other website statistics software i played with back then, so I look forward to seeing GA in action!

Activating GA was very easy, I used my existing Google account and pasted the javascript code into my blog template, just above the body-end html tag. That was it! And you can add more site watches, and administer them all from one GA account!

According to a few google searches, and a user comment on a blog belong to guy involved in developing GA, it does not seem possible to add GA to flickr sites! Like that user I find this annoying, and it could drive me for a switch to Picasa, which most like will get GA functionality before flickr!

Search + single point of entry + availability = succesful knowledge management!

2007-09-13T10:53:00.000+02:00

In my very first post on this blog i mentioned the importance of search systems/capabilities when you want to have a successful knowledge management system. Forget about categories or sorting and agreeing to one format for all knowledge, I predict it will not work for you if you go down that road! Instead think multiple systems and formats for storage, and focus on single point of entry for availability and search! Does this sound like something you know? Google! It is not without reason that Google "won" when compared to old indexing search sites!

I have two agendas for this search tool/search system/search engine investigation: I am looking for something useful for an enterprise and on the other hand I want to check out the open source posibilites so I can have something to play with at various home/friend projects! The main differences is money and how many systems/data sources the search can crawl/index and interface to. No matter which agenda you have, you should be able to get inspired from this list of requirements:

Index ViewVC websites, which can be protected by shared login credentials.
Crawl text and pdf documents on websites.
Must scale well for many documents!
Must be gentle/tunable and handle errors gracefully.

And this nice-to-have feature list:

Administration of who(users, public, ip-based) has access to search information from different sources.
Index/search multimedia formats, pictures and video, similar to Blinkx and Google images.
Handle searches with foreign charsets, eg. danish æøå.
Crawl docs on FTP sites, eg. anonymous login.
Crawl new and old Microsoft Office documents, such as Word, Excel and Powerpoint.
Crawl Windows shares.
Crawl WebDav.
Interface to and crawl Microsoft Sharepoint sites.
Interface to and crawl Lotus Notes databases, at least through web enabled databases.

I started by looking at Creating Google Custom Search Engines (Google CSE) and Google Custom Search Business Edition (CSBE). These are not free services for the requirements I have, so I have decided not to spend more time with these. Snips from the Google CSBE website:
Custom Search Business Edition is great for public websites that have a lot of web-based content that needs to be easily searchable.

Google desktop search also does not fit what I am looking for, so moving on.

Google has some other products which looks very interesting, Google search appliance (GSA) and Google OneBox. OneBox can supposedly interface to many systems (CRM, ERP, etc) and you can get your own developed module. Take a look at the different GSA products, or use the feature matrix for the different versions of GSA. GSA or OneBox is definately very interesting, especially for the large enterprise, who might want to save ressources and spent some money to get what is probably the best search tool in the world! But I dont have any of those Google tools availble to me right now and probably never will, at least not for private or community usage!

So I kept searching ;-) and I quickly became fond of the incredible details and amount of information available at Search Tools for Web Sites and Intranets (http://www.searchtools.com/).

I found several open source search tools which seems to fit a fair amount of my requirements and nice-to-have features above, so I would like to give the folllowing a try: OpenWebSpider, ASPSeek, mnoGoSearch, DataParkSearch and Swish-E.

It was not crystal clear to me which could in fact index Microsoft Office files, but at least Swish-E and ht://Dig seemed capable.

I owe to say that I tend to stay away from Java and PostgreSQL based systems as I have little or no experience in running those for a while!

Some of the open source search tools are available in the FreeBSD ports collection (of which I am a huge fan) so those will be the ones I test: DataParkSearch, Swish-E and mnoGoSearch!

Other urls I visited during this initial search tool investigation:
http://www.searchenginewatch.com

What is a search engine? http://www.techweb.com/encyclopedia/defineterm.jhtml;jsessionid=DB3VMBYCAINF4QSNDLRCKHSCJUNN2JVN?term=search+engine

Read about Wikia, see:
http://www.informationweek.com/blog/main/archives/2007/08/will_google_be_1.html

Comparing files and folders

2007-09-12T15:23:00.000+02:00

When it comes to comparing files, I have been used to a perform this on text files only, always wondering if similar functionality is availble for Word documents or even images. The tools i have used in the past:

diff -y --suppress file1 file2, very usefull for checking changes to config files, eg. in scripting if you want to make only a certain number of lines are changed. Adding something like MKS Toolkit or GnuWin32 will give you similar tools on your Windows platform.
Total Commander has a built in compare files.
My favorite is WinMerge which is freeware and can recursively compare folders!
WinMerge can also magically replace the builtin side-by-side compare functionality of TortoiseCVS and TortoiseSVN which are my favorite Subversion and CVS version control Windows interfaces.
FreeCommander can compare files if you set it to use WinMerge. Without WinMerge FreeCommander can recursively compare files and folders (Syncronize).

I dont recall needing to merge a lot of files, so i dont know how the tools above will compare to Araxis Merge.

So at my new job i was pleasently surprised to learn of Araxis Merge, a new tool, that also can compare pictures! I do not have any particular use for it right now, but i am sure i will think of something :-) Unfortunately it will cost me some bucks, so i will probably stick with WinMerge for the time being.

Blog backup and statistics?

2007-09-12T14:43:00.000+02:00

After writing my first blog post I accidently deleted parts of it! What happened was this:

I edited the blogpost, saved and published it, so far so good.

Just after saving it, i got an idea for a minor change, so I went back using the browser, to the editor windows, this was a mistake! I did not notice the text was the old text before i changed it in the first place. So when i published what i thought was a minor change, in fact the first major change was gone!

So after retyped the first major change to my posting, it came to my mind: how do i perform a backup of my blog here at blogger.com?

Also I would like to know if someone actually visits, maybe there is some stats similar to that of AWStats?

Starting a blog, handling knowledge management

2007-09-12T12:32:00.000+02:00

Welcome to my blog, thanks for visiting!

I have started this blog to improve my knowledge management system! In short this blog will contain all information i feel like saving! For more details of the entire system, see later.

The need for an improvement to my knowledge mangement came up this month, when I got a new job! At my new job I can no longer commit/checkout my personal Subversion or CVS repositories. And I dont have access to Firefox, so I am also missing my bookmark sync-and-sort plugin!

Things you wont find here are real personal information or notes that are confidential, which will have to stay on my PC or in a special Subversion repository for that.

So to summarise my knowledge management system as of today, it consist of the following:

Ideas/readme/snip commits used to be saved in Subversion, but probably this will go into this blog from now on!
Personal scripts will still go into personal Subversion repositories, as it is easier to deploy to servers. Snippets from those will go to the blog when appropriate.
Howtos/working notes probably will stay in the appropriate CVS/Subversion repositories for a while. This is not optimal for sharing with more than a few people, so snippets will be in this blog!
Pictures go to appropriate flickr accounts: personal or family, available to anyone, family or friends.
Videos unfortunately can not be put into flickr. A place like flickr, with video power like youtube would be nice! Any ideas.
E-mail will probably move more and more into gmail, as that will hopefully be availble anywhere i ever need it.
The few websites i help o webmaster, are saved in a Subversion.
Instant messaging logs are not central or searchable, this would be nice to see.
I dont contribute to any particular Wiki anywhere, neither do I have one of my own.
I dont contribute to a particular forum, neither do I have one of my own.
I have not yet started using VoIP or mobile technology beyond low-tech personal use.
Daily top urls to visit (bookmark management) will stay in Sync-and-sort for now, but should not grow into a mess like recently. Instead i will post on this blog, including my thoughts of a particular url. I have a few ideas for better bookmark management so i dont have to use sync-and-sort.
Book reviews and notes will move from Subversion to this blog.

Search capabilities within all systems is of great importance, and if I was to share knowledge i tend to say good search possibility is the most important requirement of a knowledge system! Otherwise you will risk the system never gets used.

For my own setup above, a generic search across all systems is not available to me, I have to search each of the knowledge system parts in what ever I can. This is one of the reasons i prefer any format that is text based, because then at least i can grep for one word. I would really like to have a single point of entry search engine which can crawl any of the above! Limiting access to see and perform searches within certain data would be paramount! I am not aware of a product that can do this. For the enterprise at work we will take a look the Google search appliance, but for my personal usage i hope to find something similar that is available in some open source project?

The IBM quickr approach is appealing to me, at least from a coorperate knowledge sharing point of view. It seems perfect for Notes environments, but unfortunately i have not had a chance to try it out yet! I wish there was an open source alternative with similar functionality i could play with. A google search got me to Sun portal server but it 1) it might not be what i want and 2) has some pretty hard technical requirements for me to get started, so i will probably never know about the first issue.

I dont know how other technical people cope with the difficulties of handling job and personal knowledge management systems? Undoubtedly it must raise problems with regards to people loosing their notes if they change job or job position, and it goes without saying that you can not mirror work knowledge mangement systems off for your personal usage! As work and personal life keeps merging, this issue will keep popping up.