Wednesday, July 16, 2014

wget A Super Duper Image Scooper !

I love downloading images on the internet! Cars, pin up girls, scenery and of course I've downloaded my fair share of stupid human tricks and fail pix.

Occasionally you come across a page of content and you just don't feel like clicking on each image or video individually. The other day I came across a site of hot rods with over a thousand images. That's a lot of clicks ! Who has time for that. Here's what you do.

wget to the rescue !


 You may have used wget to download stuff from the internet. A simple file grab with wget would look like this :

wget http://www.websiteName.com/filename.mp3

If you've never used wget before to download a file, search the internet for a file to download, open up a terminal, type wget and paste the download link after it like in the example above.

I don't know why but for some reason when I download something from the command line using wget it seems to download so much faster than downloading it in Firefox or using a browser. So I often cut and paste download links from the internet and download with wget from the command line. Ahright, I digress ! Assuming you understand the very basics of wget, here's how we would use it to grab images from a webpage.

Note: Use man wget to learn more about this command. This is the very basics and just something I've been experimenting with. 

The Command


wget -r --level=2 -v -A jpeg,jpg --wait=2 http://www.targetDomain.com/webpage.htm

The above is all one line of code. So what do we have here and why does it work?

wget (command line utility used to download files)

-r (this is recursive, and will continue scanning directories to find the images or videos)

--level=2 (this will only allow it to scan 2 levels of directories, the higher the number the more directories it will download.)

(if you wanted to download an entire website of files you could omit the --level=2 NOT RECOMMENDED, but you could if you want)

-v  (this is verbose, and will show you whats happening as it downloads, again this is optional)

-A jpeg,jpg (this creates the accept list in this example jpeg, jpg,. You could just as easily change jpg to gif, flv, mp3, mp4 etc, etc, or change it up a bit and download jpg, and mp4. That would download images and video. You can add as many file types as you want here separated by comma.You get the point ! )

--wait=2 (this is really important, this gives you a 2 second wait before each downloaded file. This command will download files so fast, that you really want to add this to help decrease server load. If you were to download an entire site, leaving out --level you should probably increase this number to around 5 to 10 seconds. You don't want to DDoS the server.

Finally ...


http://www.targetDomain.com/webpage.htm  (the web page or website you want to download. Again I would refrain from downloading entire websites, as this can really strain the server your downloading from.)

So there you have it. This is the very basics, and you could really get more detailed in creating a Super Duper Image and Video Scooper command.  As always to learn all the command options type man wget to learn more about this powerful tool/command ! Once you create a really great command , create a script for it and then you could just run the script and add the web address for instant downloading fun.

Enjoy !

Here's a link to the ftp: man page on wget.


Just thought I'd put this right here.

I've always loved ascii art. I found this here, and just thought it would look good here. Enjoy.


Wednesday, July 9, 2014

15 Month Update Linux Full Time

It's been about 15 - 16 months since I switched over to Fedora from Windows 7. I have to say, the longer I use Fedora full time the happier I am with my decision to drop Windows all together. The more I dive into using Linux the more useful stuff I discover. Much of this I will start sharing on the blog.

After using Microsoft operating systems for a little over 20 years, I had a real comfort factor using the product. If anything broke, I knew what the problem was and how to fix it immediately. It was an almost Zen like computing experience, and things rarely went wrong. If it wasn't for the fact that I really hated Windows 8, and the over priced software updates that came with it,  I may never have switched.

This confident user experience is a feeling I miss using Fedora. It's not for any bad quality in the Fedora product itself, but a simple lack of understanding on my part. When something goes wrong, I have to really research and read, and learn in order to fix the problem. Just like I had to 20 years ago when I started using Windows 3.11.

Just this week I  realized that familiar safe feeling is coming back with Fedora now. I may not know how to fix everything but if something is off I can usually figure out whats wrong, and what needs to be done to fix it.
I occasionally make a n00b mistake, like the other day when I deleted yum instead of a different program using RPM. Ya, I was using RPM to remove part of a package without removing all the dependencies and thinking yum for some reason and with a quick $ rpm -e --nodeps yum return, yum was gone. Dope !!! Yet, even after doing something so ridiculously idiotic, it only took about 15 minutes to fix ! Something that probably would of taken all day 15 months ago :) I have to say, it was a bit of confidence builder.

The mystery repair is always nice as well. Although it sucks not really understanding how you fixed something, and what you did to actually fix the problem. The fact that you got it to work, and knew what you had to do to make it work, is always gratifying. When this happens it's just a matter of time before you understand the "why" and "how" of it all !

I look back on learning Windows 20 years ago and remember how amazed I was with the computer. I read everything I could get my hands on, and learned the system inside and out. I could tell you what files did what, and why Windows used that file. I want to get this feeling with Fedora and Linux in general. Then I thought about it and realized 15 months into this and I've got a pretty good handle on this thing. Everyday I learn a little more now, and the more I learn the easier it is to understand. In a couple more years I'm sure I'll have that zen like feeling using Fedora or any brand of GNU Linux I choose. In the mean time I have a ton of great open source software to use, and a million ways to tweak my system for maximum work-flow.

On a final note, that is one of the things I'm getting really comfortable with now. I love the way a system can be tweaked a dozen different ways depending on what you're doing. Things like multiple desktops, and setting up the alt tab Windows switching are all huge time savers. The more I learn about KDE and the Linux command line, the more I incorporate the functions into my daily work-flow, and the more time I save. This in itself is worth the price of admission and will probably be the reason I never switch to a different operating system again. Only time will tell.