TECH: Vampiring a website database

Thursday, July 20, 2006

For many reasons, I often want to vampire an online site. I first tumbled to the concept when a certain online resource that I was using for my alumni research activities went offline for good. I was kicking my self for not having a copy. So I developed the technique of vampiring a website manually in a very organized fashion into a local copy. The problem occurs when you want to stay in sync sort of with it. That is can you come back to it a month later and find the adds and deletes. Changes are a much more complex problem. So here’s how I do it for the XYZZYX website.

(1) I use a spreadsheet to track the “database”. The first column is the web page sequence number and the second column is the entry number on the page. So the Fifth Sequential Page Seventh Item referred to Jones, I would represent it as “5”, “7”, “Jones”, and anything that was important to me.

(2) I’d open the Jones subpage and copy it to a text file (much smaller than HTML) saving it in a directory XYZZYX. If it’s going to be a LARGE number of entries, I may create 26 subdirectories A thru Z.

(3) I’d go thru the website db capturing data. When I revisit the database, since some most are too big to suck out at one sitting, I can then check key pages to see that the last entry on a page still lines up with my sequence number.

(4) If it lines up, I resume where I left off.

(5) If it doesn’t line up, I split the difference and check that page. In a 500 page database, I see I am out of sync when I try to resume at page 450. I’d go check page 275 and see if I was in sync there. When I find the discrepancy, I insert a new line in my spreadsheet for the new data and adjust columns 1 and 2.

Is this ugly, yup. Does it work? Sort of. It is posible to keep tabs on website data bases if you have an organized approach.

When you come back to look at it, you can spot check as above.

It is ESSENTIAL that you have an organized approach. It’s easy to become confused and frustrate yourself.

Questions?


TECH: “OUTLOOK” does a lookout again!

Thursday, July 20, 2006

(1) Apparently, LookOut (Microsoft’s Outlook’s evil alter ego) doesn’t honor the don’t send until flags. Argh!

(2) Apparently, Microsoft in its infinite wisdom has decided that now before I can click a link in an email, I have to say ‘mother may”. That’s is it turns off all links and it takes two clicks where one would do! Arghhhhhh!!!!


TECH: Understanding the value of RSS as opposed to email.

Wednesday, July 19, 2006

http://www.llrx.com/features/rss.htm

The Future of RSS – Is E-Mail Publishing Dead?
RSS-based Information And News Feeds: Pros and Cons For Content Distribution Through RSS
By Robin Good
Published September 29, 2003

Robin Good (Luigi Canali De Rossi) is the Editor of Sharewood Tidings, the alternative and independent news source on the effective use of new media technologies for online communication and learning.  He is the author of several web guides, including Robin Good’s Mini-Guide, Become A NewsG-d.

***Begin Quote***

E-mail is a two-way communication medium while RSS is only a distribution one.

***End Quote***

To that, I would add that IM is email in real time.

This is still an excellent summary of the value of RSS. Or, distribution as opposed to communication.


TECH: Determing age for legal purposes to a legal certainty!

Monday, July 17, 2006

http://tinyurl.com/lmup3

Jul 17, 2:10 PM EDT
Online Age Verification May Prove Complex
By ANICK JESDANUN AP Internet Writer

***Begin Quote***

NEW YORK (AP) — At MySpace.com and many other popular online hangouts, a 30-something woman can celebrate her Sweet 16 over and over with just a click of the mouse. A 12-year-old can quickly mature to meet the sites’ minimum age requirements, generally 14, while an adult looking to chat with teens can virtually shed several years.
***End Quote***

It would seem that the ISPs know who to bill for the cost of an internet account. That consists of a primary and zero or more secondary ids. It would seem that if there was a secure protocol to establish and transfer that knowledge this could be a solved problem.

Let’s assume for a moment that the unit of identity is the email address. The email address is defined by user name and ISP name separated by an at sign. A web site could inquire of the ISP does this email represent an adult, a teen ager, or a child. The ISP would have to provision the status of the email address by a query to the primary id. Ideally when the primary permits the secondary account it could be characterized at that time. I’d envision: adult, tween (i.e. between 13 and 18), and child (i.e., under 13).

To prevent spammers from using the service, the service could make it a stylized communication. The web site could inquire of its hosting entity. The hosting entity could inquire of its ISP. The ISP could inquire of its peers. It’s not a high volume transaction requiring fast turnaround.

SO a spammer could NOT say verify live email addresses or derive live address by submitting every address in a name space.

I’d also envision some type of micropayment per inquiry. Let’s say the inquiry costs the website a penny for the hosting company, a penny for the first ISP, and a penny for the final ISP. ISPs would surely net. It’d all be electronic so it would just be to ensure fairness.

Then web sites could establish reliably legally the age of a user presenting an email. The ISPs could help protect their Customer’s secondary users.

This represents my opinion and not that of my employer. I don’t design protocols for living; I’m just a philosopher. I’ll defer to the experts for feasibility.


TECH: “COMMONGATE” a free social site exposes another users info!

Monday, July 17, 2006

After registering, going to the account page shows you someone else’s info. Doesn’t bother me because I use different stuff at different sites!


TECH: “GMAIL” … is unavailable!

Monday, July 17, 2006

On Mondat 2006-07-17 at 0445 edst! I guess Google Gmail has some warts despite what everyone thinks.


TECH: “OPENDNS” … an interesting alternative!

Saturday, July 15, 2006

http://www.broadbandreports.com/shownews/76218

OpenDNS
Everyman DNS services
Posted on 2006-07-10 17:37:27

http://www.opendns.com/

Kind words
“OpenDNS has done an amazing job running this free Internet resource. It’s a very impressive service built by experts in the field.”
—David E. Weekly, founder, PBWiki
****

Worth trying.


TECH: “CHOICEMAIL” definitely gets my “don’t bother trying” rating!

Saturday, July 15, 2006

My free trial of  CHOICEMAIL ended after thirty days. Remember it was pitched as “free for one account”. Guess it isn’t. Cause it just stopped working. Then, when you refuse to spend 59$ it picks up it’s marbles and goes home. That is it doesn’t restore your old email settings. AND, it doesn’t allow you to leave email on your email account forever and ever, it deletes it upon receipt. Arghh! Well, from free or beta software, I don’t expect much, or even that it’s housebroken, or even a well mannered guest. From a commercial product trying to convince me to buy, I expect a lot more. On a picky detail note, the product never figured out no matter how many times I told it, that YAHOO GROUPS was OK, not a spammer, and to pass it right along. So, I definitely will return to the trenches fighting spam on my own. I’ll just pray to Great Blue Frog Spirit in the sky for the coming of the Giant Blue Frog to save us all. Maybe that is IPv6!


TECH: My “Bat Line” for urgent contact

Friday, July 14, 2006

http://2idi.com/contact/=reinkefj

People have expressed interest in my “Bat Line” for urgent email contact.

I use different email accounts for different purposes and handle a LOT of email. So, it would be nice to have a spam free channel for urgent emails. Outlook rules move mail to the correct account if someone uses an odd one wrong.

A gmail id was created just for this purpose with one of my “patented” email addresses (i.e., long random gibberish string for the name). So it’s unguessable or find-able by the alpha spammers. I also use
But, friends, Customers, and others who know my system can get my immediate attention by putting a message in via this mechanism.

If I’m on any one of my computers, then the IM client tied to that email address will fire and alert me of “incoming” on that special account. I attend to anything coming in on that one ASAP.

It’s my signal to don my cape and save the world. :-)  At least in my own mind!

Questions?


TECH: Feds can force you to a video?

Thursday, July 13, 2006

DRUDGE REPORTS

http://www.breitbart.com/news/2006/07/12/060712191204.9i5b29n0.html

US unveils emergency alert system for mobile phones, computers
Jul 12 3:12 PM US/Eastern
***Begin Quote***

The US government unveiled a communications system that in case of emergency should soon allow it to send SMS alerts to Americans’ mobile phones and computers.

“We have the ability to do this. It’s a major step,” Federal Emergency Management Agency (FEMA) Director David Paulson told reporters outside the US capital as he unveiled the program’s design.

The Digital Emergency Alert System (DEAS) will include the participation of television networks and public radio stations and be based on an existing alert system built in the Cold War era for use in the event of a nuclear attack.

*** AND ***

Internet-linked computers will automatically switch on to a video message from the US Department of Homeland Security while downloading instructions prepared specifically from natural disasters, chemical and nuclear attacks, and other calamities.

***End Quote***

HUH! What else can they do? And, how are they doing that. More importantly, what else can they do with “my” equipment without my permission. Arghh!


TECH: Voting Machines are not as good as an ATM!

Tuesday, July 11, 2006

http://www.lewrockwell.com/bovard/bovard30.html

The Fraudulent Meaning of Elections
by James Bovard

***Begin Quote***

Rep. Maxine Waters (D-Cal.) complained that many states used “more sophisticated technology” for lottery tickets than for elections:

<Quoting Waters>    Incredibly even in those few jurisdictions that have moved to electronic voting … we do not require a verifiable paper trail to protect against vote tampering. If an ATM machine can give each user a receipt that that user can rely upon, then a voting machine should also be able to give a receipt.
***End Quote***

Now I’m no fan of any politician, but it seems obvious to me.

(1) The voting machine (a computer, whose pedigree and programming is unknown, thus deemed insecure) needs to produce a paper record that the voter can verify.

(2) We don’t want people to be able to “prove” to the ward heeler outside the poll that they voted for the “right” candidate and paid for their vote. So that peice of paper needs to be “cast” into a ballot box as the official record of the voter’s intention.

(3) Recounts could then be done if the machine tally is suspect. (Since it’s basically an ATM receipt, it MUST be printed on non-thermal paper.) Recounts of the official box should match the machine totals.

Seems obvious to me, but what do I know!


TECH: Wifi should be like “free local calls” … … just expected!

Tuesday, July 11, 2006

http://ricksegal.typepad.com/pmv/2006/07/four_points_1_e.html

Four Points 1, Embassy Suites 0

***Begin Quote***

So, I’m in the room and turn on the laptop.  There is a fixed line net connection in the room for the low low price of 8 bucks plus tax. No wireless except downstairs in the lobby which you have to pay for as well.  Not one to just click bill the room and having left my EVDO card at home, I look for available WiFi.  Right across the street is a Sheraton Four Points Hotel and their free WiFi is bouncing her way over here to my room on the fourth floor. Sweet.
***End Quote***

The question is “where will you stay next time”. If it’s not Four Points, then Embassy Suites slid by.


TECH: BACKUP MONDAY, done!

Tuesday, July 11, 2006

As decided yesterday, I did a backup of my lovable LUGGABLE laptop. (It’s one heavy sucker!) My Western Digital Passport Drive that I snagged at Sam’s awhile ago for a little under a 100$ was the magic charm. I found out from the blogosphere that WD had it’s own sync software, which after a few high jumps, I was able to download. I tried yesterday during the day, but it takes a while to run and I have little patience. So last night, before retiring to sleep, I set it up and started it running. Seemed to go AOK. Tonight, I’ll run a restore to an old pos pc and let’s see what happens. After BACKUP MONDAY can’t be a success unless and until RESTORE TUESDAY works OK. Now what can we do on Thursday, siesta?


TECH: You have to just love what the inet can do … … make you laugh!

Monday, July 10, 2006

http://www.gobelins.fr/galerie/animation/gen2006-1.htm

I’m not sure I understand it. But, I did think it was funny!


TECH: BACKUP MONDAY, joins my “line up”!

Monday, July 10, 2006

Similarly to when I created “wireless wednesday”, I am “creating” Backup Monday.
Wireless wednesday was my short hand way of scheduling one day a week that I would take VWBIIE out and exercise it.

As an old IT guy, I know that assumptions are silently invalidated by the universe. Like the old ture-isms, “diversely routed networks aren’t”, “disaster recovery wont”, and “you never find the last bug in your application”, “backup doesn’t”, one has to aggressively prove that stuff works.

It’s an article of faith in the IT “religion”, that you can leave something working, or have something that works flawless every time you try it, but as soon as you turn around, depend upon it, or try to demo it — typically to a Customer or your boss’ boss — it will appear as if it never ever worked. Evil spirits will come from far and wide to sprinkle bad luck upon you and ever device you then touch.

So, on Mondays I will focus on “backups”. Take them and off-site them. Test them. See what works and what doesn’t.

One has to try to defeat those evil spirits any way you can.

When was the last time you used a backup? Care to be on it working? Just tell me if you need it to or not. ;-)


TECH: GOOGLE puts a new better version of PAGES out

Sunday, July 9, 2006

http://jxymxu7sn5ho9d.googlepages.com/home

Neat way to throw up pages.


TECH: “PLAXO” doesn’t like certain email addresses

Friday, July 7, 2006

From a recent email:

“FYI, I don’t know if you’ll care, BUT, Plaxo doesn’t like your email address. If Plaxo doesn’t like it, I’m sure that it falls afoul of other stuff like spam filters. I prefer, for business purposes, boring old email addresses like LASTNAME_FIRSTNAME_MI @ A_GOOD_ISP or GMAIL or YAHOO. FWIW ymmv, fjohnr”

And here’s what Plaxo regurgitated

***Begin Quote***

The following messages were not sent because the recipient addresses were identified as likely to be distribution lists. For privacy purposes, Plaxo blocks messages sent to group addresses like all@xyz.com or xyz@yahoogroups.com.

***End Quote***

Forewarned is fore armed!


TECH: VWBBIE down the shore is a little slow today.

Monday, July 3, 2006

Download Speed: 126 kbps (15.8 KB/sec transfer rate)
Upload Speed: 1284 kbps (160.5 KB/sec transfer rate)


TECH: VWBBIE can compete on speed

Saturday, July 1, 2006

FROM MY LIVING ROOM
AOT that parking lot I was in the other day!
VWBBIE

Download Speed: 103 kbps (12.9 KB/sec transfer rate)
Upload Speed: 329 kbps (41.1 KB/sec transfer rate)
WIFI2CABLE

Download Speed: 3651 kbps (456.4 KB/sec transfer rate)
Upload Speed: 353 kbps (44.1 KB/sec transfer rate)


TECH: VWBBIE and LUGGABLE come thru

Friday, June 30, 2006

Yesterday, a lot of my technology delivered nicely.

LUGGABLE is a power pig. Without an umbilical power cord, it will only run for at best an hour. So, recently I bought a auto lighter to ac plug at WalMart labeled Black&Decker. It says a 100w. I know that LUGGABLE insists on 130w to boot from the ac adapter. If the plugs not in all the way or crimped in a certain direction, ng.

So yesterday morning, I “did” my morning scrum for work by cellphone from a parking lot.. Their blackberry cellphone was strangely out of power so I had to use mine personal one. Arghh!

Then, I booted LUGGABLE from the battery. And then plugged it into the cigaretter lighter. It worked.

I went up on VWBBIE, after fighting off all the public wifis who wanted to service me. I should have tried them, but I was not into lab work. I had deadlines.

So, I tapped out my notes. Put them into the corporate email. Watch them come, via bcc, into my retail email account dedicated to work.

Shut down everything. Locked LUGGABLE in the trunk. And, went about my sad duties of the morning.

But all in all VWBBIE performed well. LUGGABLE was good. AND, my new BDPOWERPLUG was a great addition.

YMMV


TECH: Thinking about contact data.

Wednesday, June 28, 2006

I was thinking about the contact data that I collect. It really is disorganized much like life. What I really want is a tool for my desktop, with a web backup and access, that can vacuum in data from various sources and present different views of it. So for example, instead of having contact data locked up in MCalumDB, Plaxo, LinkedIn, Outlook, text files, and who knows where else, I want it in a standard format such that I use it when and where I want it. So for example pipe all the data to outlook in one massive file for email purposes. So for example, pipe it to linkedin for identification of potential mindmelds. So for example, pipe it to and from plaxo for automagic update. Hmm, don’t know if such can be done.


TECH: Playing with text that has to have come form me

Wednesday, June 28, 2006

This is a test of encrypted content posted to a blog and of interest to no one. When the text is encrypted with my private key, then anyone with my public key SHOULD be able to read it with the assurance that it came from me.

—–BEGIN PGP MESSAGE—–
Version: GnuPG v1.4.3 (MingW32) – WinPT 0.12.1

hQEMAwsQGBLrA6WSAQf+Ods/QIxZp9kNdFqkjjx7XGhdqcR4iNyCARI0xD3v4LnB
DyCqE0WvyU96ytHYtsBh8acqN7PWQlVc5peULBABhFEX8AD0gtisJAJ63+ks4IWX
LkgPI0J9fujg8+aM2qghH6wUqe7oBGkAHwO1hihxUVxZS4wRGmcLgvtF6EtP8ey8
QHuTqpPXC+7u/NNwIepTbqhxcwHjSSo0ZW16bUAvilIPhLUKEC6LYCETXQpGZqQz
NxepdTVHgF/Uyp5sPwR5mUVFqJbv53MpjluiOxSUzsAxevBSCwnjBGo6DHoSMgbU
sgV7Q+4ay61Ji1N3U+OUWe4rUkT89PMQRdIZKdyy6tLAFgF/miCaU2tFOxHpKIDO
ZcrTrUBLuM0CP5RGXrEtDUWjWbf0txVDTwqtVoGH+cRAYdjZJaIWVQ7XTbhxOoK9
7DI46i2j6UCvawume+449vb0AfMDO9a2L7NLtlgFxEAOT0h0skZurmOOfjFHMzja
p9RaTWM29fhJILku3pkmvJGMWd5W5LpE9goboHWSg7dSimFuVcEGsbew33bQku8J
GmFpnq6DXbUcakM6voGB5ZwzAVRBVSD9uOAfJg53HgzJZr1rSAXjlOYj5uSDc+Qa
Ena7AYjRiEk=
=2GiK
—–END PGP MESSAGE—–


TECH: LIFEHACKER provides a good “heads up” about free desktop search tool!

Monday, June 26, 2006

http://lifehacker.com/software/desktop-search/download-of-the-day-x1-desktop-search-183384.php

***Begin Quote***

Desktop search company X1 has just released their popular premium desktop search software for free.

***End Quote***

http://www.x1.com/download/


TECH: Enterprises need a centralized IT so there will be someone else to blame!

Monday, June 26, 2006

Are the Users taking over from the IT Department?

***Begin Quote***

I heard a new term this week from an experienced CIO running a good operation, well matched to the enterprises business aims, etc. He reckoned that ‘Shadow IT’ is fast becoming the order of the day as the Users, individually or in groups, enact something locally. His problem was he didn’t know if this was a good thing or a bad thing from the perspective of the business, though he certainly wasn’t too comfortable about it from a personal or professional CIO view.

***End Quote***

Bad thing! If you care about results.

Shadow IT, User computing, or the skunk works is all well and good. But, the chickens come home to roost, and the IT folks get blamed, when security, recovery, and blunders put a "below the waterline" hole in the enterprise ship.

Don't come running to me when your competitor buys a laptop with every customers' ssn on it.

Don't come running to me when the power surge corrupts your pc database that does the enterprise p&l in excel.

Don't come running to me when the alpha, beta, gamma, or bootleg software that you are running the mission critical app on goes "postal", "up in smoke", or develops the pc equivalent of an "OC7 or OC4" and you can't even find vendor, let alone get some support.

Just call India where you sent all the expertise.

Arghh. OK, beloved user, let's start putting the pieces back together. Now what is the business objective you are trying to accomplish?

There is a reason why the centralized IT is expensive. We had to pay tuition for all those lessons that you're now learning.


TECH: “MSFTWORD” … you can’t make it point to a specific html file?

Monday, June 26, 2006

Entertaining little "feature".

I have an html file. Actual lots of them. Representing alumni pages. I wanted to create a word doc html that would display their name and be clickable to their underlying page on my hd.

No problem.

Take the properties of the file, c:\yadayady\alum1.htm, and paste it to the html doc, hit the space bar, and WORD makes it a link.

Hole CNTL and tap the link.

(Should be "no prob" right? Buzz Wrong. Not in Microsoft land where all rules of common sense do NOT apply!)

You can't, or at least I couldn't make a link to anything but "C:\".

Now that's really useful! Sigh!


TECH: WALLNOTE … a todo list on your homepage … jury’s still out!

Saturday, June 24, 2006

http://www.wallnote.com/

Simple free hosted app. Valuable? Can't tell yet.