News:

In case of emergency, please visit our Lifeboat Forum, Spare Goggles.

Main Menu

New admin needed

Started by proteus, August 10, 2020, 05:20:37 PM

Previous topic - Next topic

von Corax

Sorry. No, still no response from proteus.
By the power of caffeine do I set my mind in motion
By the Beans of Life do my thoughts acquire speed
My hands acquire a shaking
The shaking becomes a warning
By the power of caffeine do I set my mind in motion
The Leverkusen Institute of Paleocybernetics is 5845 km from Reading

SeVeNeVeS

Quote from: von Corax on July 29, 2021, 12:20:27 PM
Sorry. No, still no response from proteus.

Ok, thanks..........

James Harrison

Right.  What the bl**dy b*******g hell was all that about for the last month or so, and how can we stop it happening again?  Have the keys been handed over to somebody active here with the time to keep on top of it yet?
Persons intending to travel by open carriage should select a seat with their backs to the engine, by which means they will avoid the ashes emitted therefrom, that in travelling generally, but particularly through the tunnels, prove a great annoyance; the carriage farthest from the engine will in consequence be found the most desirable.

von Corax

I am ready and willing to take on the task, either in part or in full. It merely remains to arrange the handover.
By the power of caffeine do I set my mind in motion
By the Beans of Life do my thoughts acquire speed
My hands acquire a shaking
The shaking becomes a warning
By the power of caffeine do I set my mind in motion
The Leverkusen Institute of Paleocybernetics is 5845 km from Reading

James Harrison

Quote from: von Corax on January 18, 2022, 06:55:43 PM
I am ready and willing to take on the task, either in part or in full. It merely remains to arrange the handover.

So we are still waiting for Proteus to pass things over?
Persons intending to travel by open carriage should select a seat with their backs to the engine, by which means they will avoid the ashes emitted therefrom, that in travelling generally, but particularly through the tunnels, prove a great annoyance; the carriage farthest from the engine will in consequence be found the most desirable.

von Corax

Quote from: James Harrison on January 18, 2022, 06:58:37 PM
Quote from: von Corax on January 18, 2022, 06:55:43 PM
I am ready and willing to take on the task, either in part or in full. It merely remains to arrange the handover.

So we are still waiting for Proteus to pass things over?
Pretty much, yes.
By the power of caffeine do I set my mind in motion
By the Beans of Life do my thoughts acquire speed
My hands acquire a shaking
The shaking becomes a warning
By the power of caffeine do I set my mind in motion
The Leverkusen Institute of Paleocybernetics is 5845 km from Reading

J. Wilhelm

Quote from: von Corax on January 18, 2022, 06:55:43 PM
I am ready and willing to take on the task, either in part or in full. It merely remains to arrange the handover.

What if we, as Uncle Bert suggests, find the FTP backdoor? Maybe that'd facilitate raw data rescue.

Lazaras

#182
Is there any way to scrape the entire forum? Far less elegant than just going through and getting the keys for all sorts of reasons ranging from none of the login credentials being preserved all the way up to mangled data...

But.... just in case, it might be a good idea to see if anyone from archive.org can help out.

Edit:

Emailed info@archive.org the following

Long and short is this,

The forum (http://brassgoggles.co.uk/forum/) has been up since at a minimum 2006, probably a little longer? Thing is nobody on the forum can get hold of the guy who has back end access and the latest outage lasted close to a month. So everyone's on edge and while hoovering the whole works and trying to build a new forum would be inelegant and a pain in the neck for all manner of reasons, I'm trying to figure out options, so 'Hey ask the guys that make archiving things their personal business if they have any suggestions.'

Note: I am not staff or of any special importance. I've just been there for the past fifteenish years and would hate for the whole thing to get tossed in the digital woodchipper.
Cheapie Theatre - Web Proxy for my Gemini Site
Want something to read? Got ten minutes to kill? Here you go!

Spare Goggles
If this forum goes down. I shall be reachable there under the same name.

Sorontar

Fair question (and email) Lazaras.

We could actually scrape the contents of the forum at least. The problems here are:
1) Someone would have to write the software to scrape it
2) We would have to have an archive to store the scrapped data in
3) Someone would have to parse and wrangle the scrapped data into a format that is useable.

For each of those:
1) doesn't seem to be too hard -> http://brassgoggles.co.uk/forum/index.php/topic,18.msg will get me a thread from 2007. The main issue will be working out how to get articles from multiple pages for each topic.
2) Not sure on how much data this would be. This is topic 51232 - that's a lot of topics but they won't be much data.
3) This is more effort. I am not sure what format would be best but parsing the html pages is possible. It would require things like
 a) extracting metadata about the topic, like how many posts, when was it scraped, what was the number of the last post
 b) extracting each post, its metadata and its content
 c) identifying anything else crucial from each webpage

This is standard sort of data science stuff for text data mining from webpages, but it still is a lot of work. The bigger issue is that this is designed for a once-off scrape and it would take a while to run (once it was made). This is not something you would want to do each day, unless you had some way of knowing what new content had been added, so you don't repeat what you have done before.

So scraping is possible but not very efficient.

If we had access to the database, things would be a lot easier.

The truth is the big issue is still
1) what happened in Dec 2021 that "killed" GB?
2) what happened in Jan 2022 that brought it back?

Sorontar
Sorontar, Captain of 'The Aethereal Dancer'
Advisor to HM Engineers on matters aethereal, aeronautic and cosmographic
http://eyrie.sorontar.com

Prof Marvel

Quote from: Lazaras on January 19, 2022, 05:33:04 AM
Is there any way to scrape the entire forum? Far less elegant than just going through and getting the keys for all sorts of reasons ranging from none of the login credentials being preserved all the way up to mangled data...

But.... just in case, it might be a good idea to see if anyone from archive.org can help out.

Edit:

Emailed info@archive.org the following

Long and short is this,

The forum (http://brassgoggles.co.uk/forum/) has been up since at a minimum 2006, probably a little longer? Thing is nobody on the forum can get hold of the guy who has back end access and the latest outage lasted close to a month. So everyone's on edge and while hoovering the whole works and trying to build a new forum would be inelegant and a pain in the neck for all manner of reasons, I'm trying to figure out options, so 'Hey ask the guys that make archiving things their personal business if they have any suggestions.'

Note: I am not staff or of any special importance. I've just been there for the past fifteenish years and would hate for the whole thing to get tossed in the digital woodchipper.

Greetings Laz...
Any reply from way back team?
Prf mumbles
MIGRATION to Spare Goggles under way

yereverluvinunclebert

#185
Hello. Welcome Back.

In my experience as a forum host there are many add-ons and extra functions that require regular administration in order to clean up temporary files, log files, .tmp files left over from captcha, processed images take up a lot of space and these need tidying.

If the site was cached, then possibly a scheduled CRON job came along and cleared that cache, freeing up the space and allowing the underlying PHP software to operate again. It seems likely that some sort of CRON task was carried out, probably removing temporary files left over from image manipulation for embedding images or similar.

If the problem was server-based rather than site-based then a server reboot might have kicked something, a process perhaps or a configuration problem and the site was raised back into life.

It could be regular backups being stored on the server using all the space. Backups are big. Perhaps a backup process created a large backup and froze with not enough space. A reboot removed the backup and the allocated space.

Any of these could have been the problem/solution. If the forum software needs free space in order to function then the site has free space again. However, although this is lovely, the same scenario is likely to occur again and possibly more frequently now.

-oOo-

If we knew the host, we could ask the host to allow us to raise tickets when such problems occur even though we are not the actual site owner. A friendly host may respond well to that and allow an admin to do some clean up as long as the work is trivial and not onerous. If the work requires payment, they might also allow another account to pay on the site's behalf. Someone here could take it on themselves to identify where the site is being hosted, open an account with them and start a conversation with the host.

The FTP details would be useful. If there is a full backup of the site somewhere then we could use that. Backups of the database might be present and then we would have the core data and would be able to restore the db elsewhere minus images and attachments. The images might be located in an image folder and then a bulk download could save those. Knowing the forum is based upon Simple Machines technology means that we could use the backups and recreate the BG forum under another domain name. I have already suggested an alternative and we NEED to buy those soon.

NO DELAY.

The vital thing is to make progress on a committee, get the members, make a quorum of like-minded chaps/ladies and then start the digging to extract the information.

There is some software called HTTRACK that allows trawling and downloading of the whole site onto someone's PC. It will trawl the whole site and extract every page and all the attachments to a folder on the user's PC. Do not underestimate how much space that will require. You are not downloading the site as it really is but the HTML representation of it, each page will be created as a static page that will not be updatable anymore. If this was the only way of extracting the old BG then the result would be a static site used for reference only. At least it would be saved.

I would expect a download like this to need gigabytes of space and it would take a long time for HTTRACK to complete. Only one person ought to run it. We don't want too much traffic on the site causing the problem again.

I suggest we create a group offline, find a tool to allow us all to meet online and then start the chat.
Steampunk Widgets and Icons of Some Worldwide Repute

yereverluvinunclebert

#186
At the bottom of each page you will see the text "Powered by SMF 1.1.20  Simple Machines"

This tells us that forum is in dire need of an upgrade as the latest stable version of SMF is 2.0.19. This really does have to be done as there are vulnerabilities that could exploit the forum and really feck up the database. All data would then be gone, forever. We are probably lucky that hasn't happened already.

Also "Page created in 0.26 seconds with 17 queries." seems to imply initially that the site isn't cached but that in truth that text would appear on a cached page too. It might be that non-unique requests (frequently accessed pages) are satisfied via a cache but I can't prove that. SMF does have cacheing built-in.

We need FTP details, a message from the site owner giving us access, that can be given to the current host or perhaps a cpanel username/password. With those latter details I could take a db backup, save it, take a full site backup, save it. We could then decide on the direction. Without the domain ownership the problem will come back to haunt us.

If we managed to take control of the cpanel (or equivalent) and we could take control of the site/data I would suggest buying the alternate domain name, redirect that to the current domain name and that would allow all the content to appear under either domain name. Then in the near future we can swap the domains over seamlessly and redirect the old domain to the new until the old finally dies. That may happen on 2022-09-24 when brassgoggles.co.uk expires.

The new domain name would take over before then and slowly we would get used to using the new version of the domain name. We would unfortunately lose some visibility in the search engines but it would be a price worth bearing to retain the old site in full.
Steampunk Widgets and Icons of Some Worldwide Repute

Prof Marvel

#187
Ooooohhhh Kaaaaayyyy

I have a bunch of old machines lying around.
and storage is now really really cheap.
like 5TB for a hundred bucks.
I have a couple terrabyets lying around somewhere in this mess .
and I need more disk anyway.

Soooo .


I am running wget as we speak, on this crippled little Win7 box with 100G free:

wget --mirror --page-requisites --convert-links --adjust-extension --compression=auto --reject-regex "/search|/rss" --no-if-modified-since --no-check-certificate http://brassgoggles.co.uk/forum/

it really is getting everyhting.
supposedly it will create the links from the forum index to the offline web pages. We will see.

I will let this run overnight and see what happens.

If it looks like it is working, I will hook up one of the terrabyte drives ( we like anime,what can i say .... especially stuff like Inuyasha, Natsume Youjinsho, Ghost in the Shell, Cowbiy Bebop... Oh and NHK Japan public TV! lots of great shows about japan in english that they let you
download ! ) to one of the more modern latops and let her rip...

Once i have the whole forum, i will then use the wayback machine tools to UPLOAD the entire forum to archive.org
then at least we can use sparegoggles to link to the wayback.

its not a solution
but its a viable workaround that doesnt take much human labor.

hope this helps

Current status -It is downloading the whole forum .
slowly, but thats ok

yhs
prof marvel
MIGRATION to Spare Goggles under way

yereverluvinunclebert

It is a start, I would suggest it may take as long as a day or so. I have not used the tool you are using and the last time I did this was on older machines with slower CPUs and drives. It might be quicker and it might not fill your drive but prepare for the worst.

Send me a PM.
Steampunk Widgets and Icons of Some Worldwide Repute

Lazaras

Quote from: Prof Marvel on January 20, 2022, 10:11:02 AM
Greetings Laz...
Any reply from way back team?
Prf mumbles

Sadly no word yet. I suppose I should get off my duff and see if there's a discord or IRC to get in touch with them on a more direct basis by.
Cheapie Theatre - Web Proxy for my Gemini Site
Want something to read? Got ten minutes to kill? Here you go!

Spare Goggles
If this forum goes down. I shall be reachable there under the same name.

Prof Marvel

#190
Greetz (as my friend and former Deutsche colleague Ingolf would say)

wget has been running for about 10  12 hours and has used less than 10 Gb 4GB... aprox 57,000 files so far
still going...

it is getting html versions of entire threads,and index files and all sorts of lovelies.
interestingly it is getting "printer" text only versions of threads as well.

more as we go.

I can see this thing running for a week or more.
I dont mind, it is only doing < 50k bytes at a time and maybe 130  kB/s

eg:

--2022-01-20 14:11:35--  http://brassgoggles.co.uk/forum/index.php/topic,44355.msg994471.html
Reusing existing connection to brassgoggles.co.uk:80.
HTTP request sent, awaiting response... 200 OK
Length: 7551 (7.4K) [text/html]
Saving to: 'brassgoggles.co.uk/forum/index.php/topic,44355.msg994471.html'

brassgoggles.co.uk/ 100%[===================>]   7.37K  --.-KB/s    in 0.06s

2022-01-20 14:11:36 (127 KB/s) - 'brassgoggles.co.uk/forum/index.php/topic,44355.msg994471.html' saved [47705]


more later

prf mumbles
MIGRATION to Spare Goggles under way

madamemarigold

Fingers crossed and ready to offer up offerings to the Air Krakens and any other toggle, bolt or geared entity that not only this site is stable but your downloading works so there is or can be a mirror site for when/if this black temporal hole happens again! Anything this deck-swabie can do more than willing to!

Xenos

Quote from: madamemarigold on January 20, 2022, 09:41:05 PM
Fingers crossed and ready to offer up offerings to the Air Krakens and any other toggle, bolt or geared entity that not only this site is stable but your downloading works so there is or can be a mirror site for when/if this black temporal hole happens again! Anything this deck-swabie can do more than willing to!

I didnae think to make an offering to my actual gods. I mean, it's a a minor thing as they go, and I'm not even sure what one I'd make an offering TO out of my pantheon. Odin? Probably Odin. Light some incense, have a shot, and say a prayer. Silly to some, but hey, stranger things have happened!
Don't let these shakes go on, it's time we had a break from it. Send me to the rear! Where the Tides of Madness swell, and men sliding into Hell...

Oh please don't let these shakes go on...

Prof Marvel

#193
Quote from: Xenos on January 20, 2022, 10:55:58 PM
Quote from: madamemarigold on January 20, 2022, 09:41:05 PM
Fingers crossed and ready to offer up offerings to the Air Krakens and any other toggle, bolt or geared entity that not only this site is stable but your downloading works so there is or can be a mirror site for when/if this black temporal hole happens again! Anything this deck-swabie can do more than willing to!

I didnae think to make an offering to my actual gods. I mean, it's a a minor thing as they go, and I'm not even sure what one I'd make an offering TO out of my pantheon. Odin? Probably Odin. Light some incense, have a shot, and say a prayer. Silly to some, but hey, stranger things have happened!

Note - the below is an acedemic discussion of various practices throughout the world, as oopsed to
a "religious" discussion.

As an amatuer anthropolysomething, studier of myths, legends, and religions, teller of traditional folktales and fairy tales, and amateur assistant apprentice keeper of traditional songs and oral history, I can certainly offer free advice!

Generall speaking  offering whatever one finds precious indicates a sacrifce and is said to be appreciated .

spoiler alert - below, a number of traditions are discussed in the abstract, without any judgement , prosletizing, or preference
if one is easily offended , don't read it.

Spoiler: ShowHide

If you want to get the attention of "the AllFather"  (or pretty much any Nordic entities) I have it on good authority that mead or beer and red meat is appreciated  ;)

If you want to go old school, Judeo/Christian  any meat, fruit, veg can be offered up.
new school Christian teaches offering food/alms/help/goodwill  to the poor/needy/elderly/crippled is appreciated by the above.

Jainists, Sihks, and Buddhists  recommend "good acts of kindness" and feeding the people.

Helping any sentient being that is suffering is universally seen as a positive offering.

Most if not all native americans will offer whatever food they eat, tobacco, and or sacred herbs.

Japanese Shinto, Buddhist, and Shugendo are similar.

these things can be offered up in a fire, or put out on a plate or board in an open space , perhaps under a tree.

When the spirit adressed gets what they want, then animals/birds/etc come in and chow down.

Some practices suggest that the animal that comes for it IS a manifestation of the spirit.

Some practices suggest that the spirit(s) take the "smell" or the "essence" or the "spirit" of the offering
On that basis, nice incense seems to be universally appreciated!

Once the "essence" is accepted the physical remains is up for grabs.

Shinto and Buddhists follow that method! Offerings of food left by supplicants/parishoners/ etc are ceremonially offered up to the
spirit(s) on an alter by the priests/monks/whatever , and after a short time they are taken back to the kitchen to be prepared
and eaten by the priests/monks/ the needy, or whoever else is there. Its all about appreciation and Sharing!

Native Americans have similar practices, Northwest Tribes overall "thing" is the Potlatch, Plain tribes have the Wopila .

Anything beyond that can get wild/differrent/icky/NSFW etc

And I have no idea what may or may not placate any less pleasant thingies.....



I highly advise AGAINST doing anything physical over a computer montior or keyboard, except perhaps a tish of nice incense, for entirely
practical reasons. I think we all can imagine the less harmonious outcomes from say, a Haitian type of offering on to electronics ....    :P

yhs
prof rambles
MIGRATION to Spare Goggles under way

Xenos

Oh no doubt that beer and red meat is the best way to get the attention of the One-Eyed. And if I had any of that on hand directly, I'd be taking it to our alter, but as we're snowed in, we make due with what we have by. We've entire lists of what to offer which of the various gods/goddesses we serve (oddly enough, most boil down to meats, breads, and alcohols). We've been at this a goodly while.

It's really funny when you have two pagans in the same house, both with totally different pantheons. You usually keep quite a bit of different incense, meats, cheeses, chocolates (yes, chocolates), and suchlike on hand for just such occasions, but as the past few years have been rather difficult, and the past few months in particular on us, our cupboards are a bit bare of all but the most basic items.

I do thank you wholeheartedly for the reminder though!  ;)
Don't let these shakes go on, it's time we had a break from it. Send me to the rear! Where the Tides of Madness swell, and men sliding into Hell...

Oh please don't let these shakes go on...

Prof Marvel

Ah My Dear Xenos -

I am always happy to blither on and on at the drop of a hat about almost anything!

I have operated on the principle "sharing what I have"

There are "preferable offerings" but in my mind it is hard to go wrong offering "what I eat".

Shinto Fox Deities for example, are said to be very fond of oranges. But they wont turn up their
noses at saki or just a rice ball. Especially if the supplicant explains "I'm sorry but this is all I have".


whether it is with corporeal entities in need, or the spirits I honor, I share equally and do not hold back
"the best portions" for myself .
Spoiler: ShowHide
( there is a biblical reference but I won't bother anyone with that....)


I have it on the best authority ( and several different oral histories from several different cultures to back it up)
that during hard times ANYTHING you cherish that you have to offer is seen as a "a good sacrifice" .

- Tupence from a pauper is more precious and more appreciated than a pound from a rich man

- The Lakotah have a tradition amongst the Oglala of "dog soup"  .

warning - not for the sensitive
Spoiler: ShowHide
After an incredibley harsh winter, with no buffalo and no game to be found, and the Oglala were living on pemmican and rawhide
soup, they nothing  to offer for a major seasonal ceremony . so after considerable prayer and consternation the sacrifce was made:
they decided to sacrifice one of the Dogs and make soup . It is important to understand that Dogs were important to the Lakotah, because they
were campanions, guards, and PACK ANIMALS. This was before the days of Horses.
.


- many shamanistic cultures make a point of making a "spirit plate"  to share a meal or feast with the spirits.

- there are specific ceremony examples among the Hopi, Utes, Zuni, Pueblo, Finnish Suomi and Japanese Ainu
(but I wont go into them), ... the point being that the traditional ceremonies are SO SIMILAIR it is remarkable...
about offering "only what they had".

A very devout man once said "I pray 5 times a day. But, if I miss the morning prayer, I pray twice in the afternoon"
ie: "the spirits are very understanding" .

yhs
prof mumbles
MIGRATION to Spare Goggles under way

Prof Marvel

#196
Oh Moldy Crabapples

I REALLY went off topic there, didnt I
:D :D :D :D :D

more on the wget:

up to 8.5 GB and 108,770 files
still growing.

I can click on any downloaded thread and see the individual page and discussion in its entireity, along with
usernames, blach blah blah. Its literaslly an html snapshot in time.
At this point, clicking on "next" sends me to to live BG page. When wget finishes up, it is "supposed to"
have linked in the appropriate links via secret magic index pixie dust.

and when its done ( in a week? a month?) I should have mine very own
snapshot copy of brasssgoggles everything!
to hold and view
and share with everybody
via the wayback!

::)

prof rambles
MIGRATION to Spare Goggles under way

J. Wilhelm

Quote from: Prof Marvel on January 21, 2022, 08:47:12 AM
Oh Moldy Crabapples

I REALLY went off topic there, didnt I
:D :D :D :D :D

more on the wget:

up to 8.5 GB and 108,770 files
still growing.

I can click on any downloaded thread and see the individual page and discussion in its entireity, along with
usernames, blach blah blah. Its literaslly an html snapshot in time.
At this point, clicking on "next" sends me to to live BG page. When wget finishes up, it is "supposed to"
have linked in the appropriate links via secret magic index pixie dust.

and when its done ( in a week? a month?) I should have mine very own
snapshot copy of brasssgoggles everything!
to hold and view
and share with everybody
via the wayback!

::)

prof rambles

Huzzah! Good show, Prof. Marvel!

That last property of the last indexing link connecting to a live page gives me the idea that the snapshot could be used as a "kernel" for a new forum!

Prof Marvel

#198
Quote from: J. Wilhelm on January 21, 2022, 09:12:22 PM
Quote from: Prof Marvel on January 21, 2022, 08:47:12 AM
Oh Moldy Crabapples

I REALLY went off topic there, didnt I
:D :D :D :D :D

more on the wget:

up to 8.5 GB and 108,770 files
still growing.

I can click on any downloaded thread and see the individual page and discussion in its entireity, along with
usernames, blach blah blah. Its literaslly an html snapshot in time.
At this point, clicking on "next" sends me to to live BG page. When wget finishes up, it is "supposed to"
have linked in the appropriate links via secret magic index pixie dust.

and when its done ( in a week? a month?) I should have mine very own
snapshot copy of brasssgoggles everything!
to hold and view
and share with everybody
via the wayback!

::)

prof rambles

Huzzah! Good show, Prof. Marvel!

That last property of the last indexing link connecting to a live page gives me the idea that the snapshot could be used as a "kernel" for a new forum!

That would seem possible! since this mirror is all html, i dont see why I couldn't edit a bunch of links to point to "the new one".

ok, about 36 hours in we have
14.3 GB (15,457,669,120 bytes)
193,288 Files, 8,069 Folders


two consecutive file examples - under  local files /brassgoggles.so.uk/bg-forum/

index.php@PHPSESSID=4ec7d5220eb53a13750e0f9050ca5257&topic=5079.15   96.2 KB (98,509 bytes)
index.php@topic=104.msg183610                  112 KB (114,688 bytes)

the first is this complete page of a thread
http://brassgoggles.co.uk/forum/index.php/topic,5079.0.html

the second is this complete page of a thread
http://brassgoggles.co.uk/forum/index.php/topic,104.5925.html

I am hoping that wget somehow musteriously resolves the indexes going from one page of a thread to the next...

it is also saving the "printer" or text only versions of complete threads as one long file - eg

this
index.php@action=printpage;topic=11505.0                        388 KB (397,312 bytes)

is a snapshot of this thread
http://brassgoggles.co.uk/forum/index.php/topic,11505.0.html

but in this form
http://brassgoggles.co.uk/forum/index.php?action=printpage;topic=11505.0

Dang, this is actually saving user profiles and the permissions too! ... which I can't see when I open it because it is not my profile
and I don't have the permissions... hmm wait. I bet it is saving the html error page when wget "ttries" to look at the permission.

if if wget doesnt link threads in a topic, it will not be hard to write a script to do so, since each thread has its own unique topic number,
followed by a "msg" number would seems to indicate thread link ie

index.php@topic=9039.msg275269

points to this

http://brassgoggles.co.uk/forum/index.php/topic,9039.0

and I got there by manually pasting   "topic,9039" onto the the base "http://brassgoggles.co.uk/forum/index.php/"
easy to extract the topic number, and replace"topic=" with "topic,"

i used to do this cfrap all the time - we got data from one DB, we were "only supposed to be able to view it" ...
but our job required USING IT
so I exttracted the operant stuff from html or whatever, looked for links, file names, etc,
and either accessed the actual file itself or re-formated the query, or just grabbed the live raw data out of the web page
and reformatted it to suit our needs - usually to feed another program or script that acted on it.

I actually have a couple patents on programs doing just that.
One guy wrote a program to  "do stuff" and was distributed on servers literally around the world.
If it ran into problems, it would throw errors and the guy expected a humna to sit at a console/workstation
and resolve them real-time by hand.

That program worked really well at what it did, but was spaghetti code,  he ws no longer around, and
so modifying and getting it updtaed around the globe ...it would have been a nightmare.

Soooo instead of "doing what I was told" I started taking the html error data, parsing it out with "C Shell" scripts
reformatting it into a human readable dashboafrd to make it easier to deal with....
then ran basic net diagnostics to see what the fault was and where....
Then found we had a global DB of server info that I could "read" with a human realtime query...
so I wrote another script to appear to be a command line query and got the data on the failing server...
then found I could query the company phone book to get contact poop on employees...
and I rebuilt the display to show
error / server that faulted / where it lived / who "owned maintained" by name , company phone and email, and their boss

then I built anoither backend that not reply generated a "trouble ticket" with all that crap
AND the dignositic data I logged to make it easier for him to fix it!

and emailed it to the server/network admin in question
and copied his boss if he did  not reply within a couple hours.

Later I wrote another backend to put all that crap into the actual "trouble ticket system" that the compnay had.

So instead four or six of us sitting at a console all time every third day playing monket boy, we would come in at regular hours
check my trouble board, and check the emails/trouble tickets/ etc to see if the admins responded to fix it
and check my "repairs not done" board to see if it was back and we could kick off an abreviated
"redo the job" using my list of "broke but repaired servers"

THAT was very satisfying, esp since the bosses saw value and let me do what was needed. And I got a couple patents
and learned a lot about patent law. I got screwed out of one patent because IBM was pursuing something similar to one
of my tools and they filed first  :P

It was very satisfying work ....

Later we formed a small group to turn the whole thing into a real product with the intent to sell it, but we were under the wrong
VP, and some other VP in the bay area sorta squashed it by not marketing it all. No sales == dead product.
Hellfor what we had invested in it, we could have GIVEN IT AWAY bundled in with our other product stack, as a freebie wonder tool
for our customers to use themselves but oh no,  that would not fly either.

I actually had a chance to fly out on a sales tour to the PAC RIM to promote this thing, Australia, Hong Kong, Singapore, Japan...

But it started getting wierd when manager took what was supposed to be a 4-6 week trip and kept adding more and more cities...
Then he built a slide shiow presentation that claimed we were montioring and fixing over 1000 servers wroldwide...
but it was only 625. He said, "oh it will be that many by then" and I said... No I can't lie in a preso to our own peole let alone lie to
our customeres....

Sooo he said ok and got someone else to do it. They were gone for almost 10 WEEKS on a plane like every third day....
looked like hell when she finally got back....

and I moved on to another  support team under a better manager who wold not lie....

And thats what we "used to" call HACKING

oops i digressed again...

more on wget and molesting  modifying the links if need be  later

prof omg he is rambling again .....

OOOOOohhhhhkaaayyyyyy

more messing about.

the structure is becoming clearer

so we have TOPIC number == thread in brass goggles.  forum/subforum is not indicated

we have PAGE within  a topic, starting at zero and incrementing by 25 by appending said number to the topic

ie
http://brassgoggles.co.uk/forum/index.php/topic,9039.0.html
http://brassgoggles.co.uk/forum/index.php/topic,9039.25.html
http://brassgoggles.co.uk/forum/index.php/topic,9039.50.html
http://brassgoggles.co.uk/forum/index.php/topic,9039.75.html
http://brassgoggles.co.uk/forum/index.php/topic,9039.100.html

somewhere perhaps I could obfuscate the thing to put an entire thread on one very long html page...
....

and we have individual user entries into the topic designated by "msg######" appended to the topic number

ie
http://brassgoggles.co.uk/forum/index.php/topic,9039.msg168542.html

intersting

wget now up to
200,583 Files, 8,377 Folders
15.0 GB (16,122,040,320 bytes)


prf mumbles
MIGRATION to Spare Goggles under way

Lazaras

Oh.... my.

Sir i have to remove hat and salute your efforts.

And do please keep rambling. it's an instructive read. Even with that to one side it is good to document the proceedings.
Cheapie Theatre - Web Proxy for my Gemini Site
Want something to read? Got ten minutes to kill? Here you go!

Spare Goggles
If this forum goes down. I shall be reachable there under the same name.