Sysadmin sites to include in your own searchengine crawl

During the everyday life of a sysadmin Google plays a large role, but also the internal knowledge base is important as there are (should be!) cases related to your specific systems. So hopefully you are providing search for that internal knowledge, or it could go unused!!

I am thinking of collecting a set of external sites to include in our internal search engine crawling, as those sites seems to pop up again and again.

I will build a list of sites to include here, bare in mind this is a raw list, i will update it when they are actually put in the search crawler!

Windows adminstrator/script related so far:
http://www.jsifaq.com/
http://www.windowsitpro.com/topics/index.cfm?action=ArticleList&ChildTopicID=72
http://www.windowsitpro.com/Articles/ArticleID/14459/14459.html?Ad=1
http://www.windowsitpro.com/windowsnt20002003faq/
http://www.ss64.com/nt/
http://www.computerperformance.co.uk/vbscript/
http://www.robvanderwoude.com/
http://cwashington.netreach.net/
http://www.ericphelps.com/batch/index.htm
...
From ss64.com link page there are many *very* good sites, a lot I didnt know before, here some snips:

CommandLine.co.uk - Batch File examples and Utilities
FP Schultze - Batch files
OldNewThing - Raymond Chen's weblog

Heise-security.co.uk - Manage Win XP updates without an internet connection.
Timo Salmi - FAQ's - Useful NT/2000/XP script tricks and tips (tscmd)
Steve Hardy - NT/2K command line scripting
Rick Lively - Commands for every version of Windows and DOS
List of TCP and UDP port numbers

Joeware.net - Admin, AD and Exchange tools.
FP Westlake - Free Win32 console utilities.
Alexander Frink - NT Security Utils, Logoff, Change password.
Bill Stewart - Batch script and Windows admin tools.

Poor Mans SMS - scan a pre defined IP range and list all installed software.
Microsoft App Compatibility - command line tool to collect application info.

Agent Ransack - File Search for Win XP
AnalogX - Screen capture, Terminal Server copy, etc

Autohotkey - Automate keystrokes, mouse clicks.
AutoIT - GUI scripting
Bamboo Software - Scheduled Tasks and other command line utils.
DumpSec, DumpEvt - Dump Event Log, Registry or Security info.
OCS Inventory - Open Source System Management
Filezilla - FTP
Lost NT password
NTFS undelete - undelete files
nu2/Barts Bootable CDs - Admin/Recovery
Trinity Rescue Kit - for recovery and repair of Windows machines
Netikus - Password, Ping, FTP tools.
OptimumX - Utilities by Marty List
UnDelete - Diskeeper

And perhaps:

http://www.microsoft.com/technet/scriptcenter/default.mspx

FreeBSD sysadmin so far:
http://taosecurity.blogspot.com/
...

Search tools, challenges and non-trivial requirements

I have listed some key challenges for my current usage of search tools:

• Create a point of entry for search.
• Link to relevant search query from a portal (eg. a operation status website).
• Some knowledge should only be available to some people. This seems to the biggest hurdle!

Limiting knowledge/search only to some people could be solved in at least 2 ways:

1. Set up different indexer/crawler configurations, each searchable from different search prompt. Problem could be multiple crawls of the same info (load, storage, ressources)
2. Index/crawl everything once, and let the search box/website/frontend control who can see what. This would be preferred.

Listing non-trivial requirements which are not always availble:

• Parse open office word and calc, (.odt and .ods), which is basically zipfiles with xml (unzip and parse eg. content.xml).
• Crawling/indexing file sytems (shares/harddrives), setting a baseurl for how the searchresults will become browsable.
• Reindexing must automated, eg. scheduled or cron'd.

Starting a blog, handling knowledge management

Welcome to my blog, thanks for visiting!

I have started this blog to improve my knowledge management system! In short this blog will contain all information i feel like saving! For more details of the entire system, see later.

The need for an improvement to my knowledge mangement came up this month, when I got a new job! At my new job I can no longer commit/checkout my personal Subversion or CVS repositories. And I dont have access to Firefox, so I am also missing my bookmark sync-and-sort plugin!

Things you wont find here are real personal information or notes that are confidential, which will have to stay on my PC or in a special Subversion repository for that.

So to summarise my knowledge management system as of today, it consist of the following:

• Ideas/readme/snip commits used to be saved in Subversion, but probably this will go into this blog from now on!
• Personal scripts will still go into personal Subversion repositories, as it is easier to deploy to servers. Snippets from those will go to the blog when appropriate.
• Howtos/working notes probably will stay in the appropriate CVS/Subversion repositories for a while. This is not optimal for sharing with more than a few people, so snippets will be in this blog!
• Pictures go to appropriate flickr accounts: personal or family, available to anyone, family or friends.
• Videos unfortunately can not be put into flickr. A place like flickr, with video power like youtube would be nice! Any ideas.
• E-mail will probably move more and more into gmail, as that will hopefully be availble anywhere i ever need it.
• The few websites i help o webmaster, are saved in a Subversion.
• Instant messaging logs are not central or searchable, this would be nice to see.
• I dont contribute to any particular Wiki anywhere, neither do I have one of my own.
• I dont contribute to a particular forum, neither do I have one of my own.
• I have not yet started using VoIP or mobile technology beyond low-tech personal use.
• Daily top urls to visit (bookmark management) will stay in Sync-and-sort for now, but should not grow into a mess like recently. Instead i will post on this blog, including my thoughts of a particular url. I have a few ideas for better bookmark management so i dont have to use sync-and-sort.
• Book reviews and notes will move from Subversion to this blog.

Search capabilities within all systems is of great importance, and if I was to share knowledge i tend to say good search possibility is the most important requirement of a knowledge system! Otherwise you will risk the system never gets used.

For my own setup above, a generic search across all systems is not available to me, I have to search each of the knowledge system parts in what ever I can. This is one of the reasons i prefer any format that is text based, because then at least i can grep for one word. I would really like to have a single point of entry search engine which can crawl any of the above! Limiting access to see and perform searches within certain data would be paramount! I am not aware of a product that can do this. For the enterprise at work we will take a look the Google search appliance, but for my personal usage i hope to find something similar that is available in some open source project?

The IBM quickr approach is appealing to me, at least from a coorperate knowledge sharing point of view. It seems perfect for Notes environments, but unfortunately i have not had a chance to try it out yet! I wish there was an open source alternative with similar functionality i could play with. A google search got me to Sun portal server but it 1) it might not be what i want and 2) has some pretty hard technical requirements for me to get started, so i will probably never know about the first issue.

I dont know how other technical people cope with the difficulties of handling job and personal knowledge management systems? Undoubtedly it must raise problems with regards to people loosing their notes if they change job or job position, and it goes without saying that you can not mirror work knowledge mangement systems off for your personal usage! As work and personal life keeps merging, this issue will keep popping up.