Tuesday, March 21, 2023

Travel Computer

Ten years ago, I had business in Boston for a few hours.  It's a four-hour round trip to Boston from my home, so I planned on doing some site-seeing later in the day.  Accompanying me on this trip was my 17-inch HP Pavilion laptop, a monster of a laptop that was as capable as any desktop.  But, performance comes with a cost, and after lugging a nine-pound laptop around Boston, I decided I  needed something lighter and smaller.

On our next vacation, I brought along a Microsoft RT, a slim and light laptop with a detachable keyboard, so the RT could double as a tablet.  Slim and light was great, doubling as a tablet was great, but the performance of the RT... well, not so good. I felt like I'd traded a Cadillac for a Yugo.  I brought the RT along for a couple of trips, but the performance was inadequate, and  I really needed something better.   

My next travel computer was a 10-inch Asus netbook, the X102ba.   The X102ba was great; it hit the sweet spot between the size and weight of the Surface RT and the performance of the HP Pavilion.  The X102ba was a bit slow running Windows 10, but I've been running Linux on my home computers for about three decades now, and performance with Lubuntu was very good.  

I've used the X102ba for about 5 years, and it served me well.  But last year, the battery failed, and I can't use it without plugging it in.    This is not good for traveling, when you're never sure where the next plug is.  We haven't been traveling because of the pandemic, so I was in no hurry to get another computer. I replaced the battery once; it's not brain surgery, but it's not risk-free either.   I could easily brick an otherwise good laptop.  In the meantime, the X102ba has taken on a new role, pulling weather data from  my weather station and serving the weather data on the web.   With the weather station software and Apache using my laptop, I'm on to travel computer #4.

My criteria for the new computer were
  • Light weight
  • Good performance
  • In case of theft or damage, low-cost
My Linux-powered netbook got me thinking: what else runs Linux?  My phone does.   And so do Android tablets;  they use the Linux kernel (as do many of your household appliances).  An eight-inch or ten-inch tablet is about the size of a Surface RT.   Lower-end models of tablets are fairly inexpensive and sport a reasonable amount of compute power.   Could I find tablet, purchase a bluetooth keyboard, and use it as my travel computer?   

In this post, I'll discuss my experience using a tablet in place of a computer.  Of course, the tablet is a computer, it's running the Android OS rather than Ubuntu or Windows or Mac OS.  Android or IOS or Mac or Windows or Linux or your TV; they're all computers.   For brevity's sake, in this post when I say computer, I'm referring to desktop computer or laptop computer running Windows or Mac OS or Linux. When I say tablet, I'm referring to an Android tablet.  So, this is the  issue:   Are tablets fast enough, and is the Android operating system and its apps flexible enough to replace a Windows or Linux  or Mac computer?   I will take a task by task approach, comparing my experience using the tablet to using a desktop computer.

My test bed is a Samsung T295 8 inch tablet.  Let's discuss this tablet first.   I paid less than $100 US for a refurbished tablet.  The T295 is a lower-end Samsung product. It feels solid and well-built, but the performance of this tablet is just adequate. The tablet is light, even with the added weight of a folding case and a Bluetooth keyboard. 

As a book reader, newspaper reader, or for streaming media, it's great.   Android apps, like Gmail, run well.  And that's probably the key thing:   if the service, like Gmail, or Calendar, or Facebook, or a newspaper, has an app, that's the way to go.   If the service depends on the browser running Javascript, the performance can be sluggish.  That's the trade-off between the low-cost of this tablet and performance.  If I was to repeat this experiment, I would look for a faster tablet.

Next, let's examine the size.   My old Surface RT and the Asus X102ba had 10 inch screens, the Samsung T295 only has an eight inch screen.   And the keyboard that fits so nicely in the folding case is only eight inches, too!   As a laptop substitute, eight inches is small, both the keyboard and screen are challenging, and I really prefer a larger screen size.   As a book reader, though, eight inches is fine.

The eight-inch tablet bundles nicely with the eight-inch keyboard.   As for the QWERTY keyboard, my fingers adapted pretty quickly to the tight space.  But a keyboard this size has compromises. Some keys are not where you're accustomed to finding them . For example, the single-quote shares a key with the letter O, it's Fn-O.  And double-quotes are Fn-P.   It's like driving our three cars with manual transmissions: each one has reverse in a different position.  

I like the 8 inch screen as a book reader, but I'd prefer a 10 inch screen as a laptop substitute.   Another consideration is reading magazines.  Book readers like Hoopla or Libby resize the pages to fit the screen, but many magazine readers present whole-page images, like a PDF.   The text and pictures do not flow to fit the screen size, like HTML.  Fitting a magazine in an 8 inch screen renders the text and pictures too small to easily read; a 10 inch screen would be much better for magazines.  I can enlarge the text and move around the magazine page, but it's clumsy compare to book readers that resize the page to fit the screen.

Now, let's compare Android UI to a Window/Linux/Mac UI.  Android UI works very nicely with phones, and there's a lot you can do with gestures.  If you're trying to use a phone/tablet as a computer, there are definitely some disadvantages.   Using the touch screen rather than a mouse to highlight text, move the cursor, cut and paste are not as easy as using a mouse.   Compared to the mouse, the touch screen is pretty clumsy.  I'm still working on highlighting text for cut and pasting.  Before I publish this post, I'll review on my desktop and make my final edits.  Highlighting images for cut and paste is easier on a laptop, too. 

Let's discuss the Blogger app, since that's where I'm working right now.   The tablet app from the Play Store supports only a limited amount of Blogger functionality.   The editor does not support HTML editing; if you remember Wordpad on early versions of Windows, then you've got a good idea of the editor's functionality:   it's WYSIWYG, but it's limited.  The Blogger app doesn't work in landscape mode, either.   In addition, you can not configure the blog in any way;  you can only edit and publish posts.   If all you need to do is simple editing, attaching photographs, creating links, underline, bold, or italicize text, and publishing the completed post, then the app works fine.   Configuring the blog's theme, layout, or settings, or more complicated editing requires the web version of Blogger.  The web  version of Blogger, whether it's on the tablet or computer,  will let you do everything you need.

Email was the killer application of the 90's, and for many of us, it's still the app we use the most. Gmail on the tablet is better (faster, more polished) than the gmail web app on the X102ba netbook.   Screen to screen transition is quicker.  For email, the Gmail app from the Play Store works much more nicely than the web interface.

Other Google apps, like Calendar, Contact, Keep, and Voice also work well.  In fact, with the Voice app, a Google phone number, and a WiFi connection, I have a working telephone.   So far, Blogger is the only Google app that seems to be more limited than the web-based versions.

Also, the tablet streams video very well.  There are no jitter or buffering issues, even with a slower tablet like mine.  Pedaling an exercise bike or walking a treadmill is extremely boring, and streaming video from Youtube or Amazon Prime is a great way to help pass the time.  

Would I do it again?  Yes, it was a worthwhile experiment.   When I'm not travelling, the tablet has been very handy as a book reader.  Our public library has a great selection of books and magazines to loan for free.  What would I do differently?   I would look for a faster tablet, and I think I'd also look for a 10 inch screen.  The eight inch screen is great for newspapers and books and the many apps that were designed to run on phone-sized screens, but a 10 inch tablet wouldn't be that much larger, and I think the larger screen would generally be more useful.



Friday, February 5, 2021

Lubuntu 20.04 LTS Upgrade

After working thru my Lubuntu 18.04 notes, it's time to get started on 20.04LTS.  18.04 has served me well, but support for 18.04LTS ends in April of 2021.  With the  maintainers dropping LXDE in favor of LXQT,   I expect there will be some surprises along the way...  will things be better and faster?  Or will the new distro require more memory and processor power?       

The computer I'm grading is a vintage 2013 Asus netbook, X102ba with 4GB of memory, 320GB disk, and a dual-core AMD processor running at 1GHz, so I'm pretty sensitive to performance issues.   I need the new distrbutuon to run as well as 18.04.   And I'd like to find that the tweaks I made to 18.04 running LXDE work with 20.04 running LXQT, too .

Before getting started, I'd like to plug the Lubuntu Manual.   It's well-organized with a lot of useful information.  Many of the tweaks I made in Lubuntu releases 17 and 18 by editing configuration files can be done using the Lubuntu 20.04 GUI configuration tools.   Let's get started.

First Note: Installation Media

Everything went well through my first note, Installation media.   There are really no changes here.

Second Note:  Broadcomm Wireless Adapter

The old HP dv8210us laptop with the  Broadcomm wireless adapter took its last boot, so the Broadcomm wireless adapter is no longer an issue.  

Third Note: Consistency 

Not surprisingly, adding Windows-L (or meta-L in Unix-speak) to lock my screen has changed.  In my two previous posts, I edited the lubuntu-rc.xml file in my ~/.config/lubuntu directory.  That file no longer exists, and this time I'll use the built-in Preferences > LXQT Settings > Keyboard Shortcuts to add this function. This opens the Global Actions Manager (that's a mouth-full!).  Click the Add button, move the cursor to the shortcut and press the desired shortcut, Windows-L in this case.  Add a short comment (Lock Screen like Windows), select the Command radio button, and add the command xdg-screensaver lock.   Click OK, and you're done.   You can lock the screen with either Ctrl-Alt-L or Windows-L.

Fourth Note: Two Monitors  

There's a huge improvement here in setting the position of the second monitor.  My extended monitor sits to the right of my labtop's screen, and making everything work smoothly in Lubuntu 17 and 18 was tricky.  Now the Monitor Settings app sets everything up, there's  no fooling with xrandr,  and there's no need to add a second task bar to the extended monitor. 

Select Preferences > LXQT Settings > Monitor Settings.  Click on Set position, drag the monitors into the correct position for your setup, and click apply.   Confirm the new setting works for you, and you're all set.   

But, if you've become accustomed to task bars on both monitors (and I have), it's easily done.  Move cursor to the task bar, right click, and select Add New Panel. Set the new task bar to your favorite position on Desktop 2, then select Widgets from the left panel, and add the widgets you'd like to use.  Unlike LXDE, all the settings for both task bars are stored in one file (~/.config/lxqt/panel.conf), so there's no advantage (and probably some risk) in editing files manually.  Just configure everything thru the UI.

Fifth Note: Network and Volume Indicators

As I expected, there are changes here.  For the volume indicator, right-click on the task bar (aka panel), select Configure Panel, select Widgets from the left panel, and add the Volume Control if it's not there.  For the Network, make sure the Status Notifier Plugin (statusNotifier) is added.  The Status Notifier includes the battery monitor and the network status icons.  Everything works as it should without any tricks, so if you add these two widgets, no further action is required.

I prefer the Network indicator to the nm-tray indicator.   To change this, follow the instructions in Advanced Networking, selecting the Network and unselecting the nm-tray.

Sixth Note: More on Network Interfaces

I have two wireless network interfaces with my Asus netbook.  The built-in Qualcomm Atheros AR9485 worked fine out of the box.   In addition, I have a Realtek RTL8192cu USB dongle. When booting into Windows 10, I found the Realtek interface is much faster than the Qualcomm interface.   In Lubuntu 17, the Realtek interface would stop working every two or three minutes until I disabled/enabled the interface. Then it would work again for another two or three minutes. The Realtek interface no longer quits working every two or three minutes in Lubuntu 20.0, so this problem has been fixed. But the throughput in Speedtest is faster using the Realteck interface under Lubuntu 18.04 than it is under 20.04, so I'm going to apply the same update.


sudo apt-get install linux-headers-$(uname -r) build-essential dkms git
git clone https://github.com/pvaret/rtl8192cu-fixes.git
sudo dkms add ./rtl8192cu-fixes
# NB:  The step above reports a version.   
#      As of 2018-04-12, the version is 1.11, not 1.10.
#      Use the correct version in the next step.
sudo dkms install 8192cu/1.11
sudo depmod -a
sudo cp ./rtl8192cu-fixes/blacklist-native-rtl8192.conf /etc/modprobe.d/
sudo cp ./rtl8192cu-fixes/8192cu-disable-power-management.conf /etc/modprobe.d/
reboot

After rebooting, speedtest shows about a 10MB/second improvement in through-put.

Seventh Note: Printers

As expected, the postscript-capable Lexmark Optra S installed with no problems.  Even better, the installation found the wireless Epson Workforce 2860 and installed the correct drivers.  But installing the Epson NX420 turned out to be an even bigger headache than I expected.  

First, I had to fight through the cartridge-not-recognized messages on the Epson printer.  This is a common complaint on the web; it's just a matter of finding one of the black-magic solutions that work.   After finally satisfying the printer's lust for ink cartridges, I was able add the printer and print the test page.

The Epson Workforce installed so cleanly that I hoped I would have better luck with the NX420 this time, but no.  The test page images and text did not align cleanly.   Referring to my earlier notes, I downloaded the driver from http://www.openprinting.org  The Discover software installation tool will open the downloaded file, but Discover will not install a file from an unsigned archive. Instead of opening the the downloaded file automatically, save it, and then run the command 

 sudo gdebi epson-inkjet-printer-nx420_1.0.0-1lsb3.2_amd64.deb 

from a terminal window. There are a couple dozen dependencies that will be installed as well, so don't be alarmed. After the install is finished, right click on the NX420 printer, and open the properties. Under Settings, check the values in the Make and Model line. This looks like informational text, but it's actually the driver. Click on change, wait while Change Driver dialog. The recommended manufacturer (Epson) is already selected, click Forward. There will be two choices of drivers, choose the one that says Epson Stylus NX420 Series - epson-inkjet-printer 1.0.0-1lsb3.2. The other, Epson Stylus NX420 - CUPS+Gutenprint v5.2 is the bad driver. Select the correct driver, click Forward, click the radio button to Use the new PPD, and click Apply. The test page should print correctly now.

Eighth Note: Final Tweak

The keyboard and touch-pad on the netbook are pretty tight, and I frequently bump the touch-pad while typing.  Of course, this sends my cursor all over the place, and soon I'm typing text three paragraphs above where I should be.  Because I have a mouse, I really don't need the touch-pad most of the time, and I want an easy way to turn it on and off.

I will re-use the script in my Lubuntu 18.04 Upgrade post. The script  toggles the touch-pad on and off, and we'll just need to bind the script to a key.   Open the Global Actions Manager (Preferences > LXQT Settings > Keyboard Shortcuts). Click the Add button, move the cursor to the shortcut and press the desired shortcut key.   I use fn-F9 because that key has a small touch-pad image on it.  The Global Actions Manager identifies this key as XF86TouchPadToggle.  Select the Command radio button, and enter the full path and filename of the script in the command box.   Click OK, and you're done.   Now I can enable or disable the touchpad with the fn-F9 key.  

This completes my upgrade to Lubuntu 20.04 LTS.  By the time support of this version of Lubuntu expires, this netbook will be 10 years old.   This light-weight version of Ubuntu has added 5 years to the useful life of my old netbook. 

Thursday, January 14, 2021

Lubuntu 18.04 Upgrade

Wow.... where has this post been hiding?  I'm getting ready to update to Lubuntu 20.04 LTS and I found this mostly completed post from a year or two ago!  This information is useful to my next upgrade, so I'm going to finish this post and document a new note before starting the upgrade to Lubuntu 20.04 LTS.

So, a couple years ago I was running Lubuntu 17.04 and 17.10 on an old HP laptop and Asus netbook for about a year when Lubuntu 18.04 LTS was released.   While the 17.x distributions worked well, they were not LTS (Long Term Support) versions, and updates are no longer available.  

I decided to start with a fresh install, overwriting my old distribution.   I started with my earlier notes in Lubuntu 1704 to1710-upgrade.  Most of these notes are still useful, so in this post I'll document where things went awry.

Everything went well through my first two notes, Installation media and  Broadcomm wireless adapter.

But following my third note, Consistency, when I tried to change the Window-l key to lock my screen, it didn't work;  neither did the Lubuntu default key combination of ALT-DEL-L.   I fired up an xterm, and the command lxsession-default lock just echoed the word "lock" in the window.  Not so good.

Google is your best friend in this situation, and after googling this issue, I found that everyone is reporting it.  To fix it, we need to run a command that locks the screen.   I could install the xscreensaver command, which wouldn't be a bad thing if I was running gnome.  But I'm not, so installing 20+ dependencies to run the screensaver on hardware that is nearing the end of its life didn't seem like a great idea.

So, how does Lubuntu lock the screen?   The menu Logoff command displays a menu of logoff options, including locking the screen.  Now it's time for some brute-force detective work.   First, where is the lxsession command stored?  The command whereis lxsession shows us the executable is found in /usr/bin.   What else is stored in /usr/bin?   The command ls lx* finds the command lxsession-logout.   The command type lxsession-logout shows it's a hashed file, aka a compiled executable.  Running lxsession-logout from my terminal, I get the same pop-up window as executing logoff from the main menu.

Again, google is our friend.  Googling lxsession-logout, we find the source for lxsession-logout.c in github.  This is where open source really shines -- we'd really be stuck with proprietary vendor code in a situation like this.  Looking thru the source, lxsession-logout searches for two programs to lock the screen:  lxlock and xdg-screensaver.  If it finds one of those program, it will use the program it finds to lock the screen.  Back in our terminal session, we can look for those two programs; either key in the program at the prompt and see what happens, or search using find.  On my fresh install, I did not have lxlock, but I did have xdg-screensaver.

Now, back to lubuntu-rc.xml.   Edit the file in your favorite text editor, and after the default lock command, add the following; it's the almost the same code from the 17.04 to 17.10 upgrade, except we use the xdg-screensaver command instead of lxsession-default.
  
  <!-- Lock the screen on Windows + l-->
  <keybind key="W-l">
    <action name="Execute">
      <command>xdg-screensaver lock</command>
    </action>
  </keybind>


Save the file, and from a terminal session refresh your openbox configuration:
openbox --reconfigure
Assuming you didn't get any errors, the Windows-L sequence will lock your screen. And before you close up lubuntu-rx.xml, you might as well fix the command for CTRL-ALT-L, too.

Fourth Note:  Dual Monitors

Handling two monitors hasn't changed.   I found that I still needed to duplicate the task bar on both monitors.   This is a new feature of Windows 10 that I've become accustomed to at work, so having the taskbar on both monitors doesn't seem as odd as it once did.

Fifth Note: Network and Volume Indicators

The Network and Volume indicators seem to work a little better in Lubuntu 18 than in Lubuntu 17, but you still need to be pretty quick.  I went ahead and followed my notes from the upgrade to 17.10.

Sixth Note:  Network Interfaces

The newer Linux kernel should support the Realtek USB dongle properly in my Asus netbook. It does work, but testing my network speed with my ISP's speedtest showed that the performance was about half that with Windows 10.    I followed my notes from the earlier upgrade and used the better driver.   Performance is now slightly better than Windows 10.

Seventh Note: Printers

The Lexmark Optra S installed with no problems, as expected.  But installing the Epson NX420 turned out to be a bigger headache than I expected.  As expected,  the default installation did not work properly.     Referring to my notes, I downloaded the driver from openprinting.org.  As before, I chose to open the file with the Software Center, but nothing happened...  Google has now become my best friend,  and there are several reports of the latest Software Center not processing .deb files correctly.

Luckily, reports of the problem also included suggestions to use the command line tools.   Instead opening the the download automatically, save it as a file, and run the command

sudo gdebi epson-inkjet-printer-nx420_1.0.0-1lsb3.2_amd64.deb

There are a couple dozen dependencies that will be installed as well, so don't be alarmed.    After the install is finished,  right click on the NX420 printer, and open the properties.   Under Settings, check on the Make and Model line.   This looks like informational text, but it's actually the driver.  Click on change, wait while Change Driver dialog. The recommended manufacturer (Epson) is already selected, click Forward.  There will be two choices of drivers, choose the one that says Epson Stylus NX420 Series - epson-inkjet-printer 1.0.0-1lsb3.2.   The other, Epson Stylus NX420 - CUPS+Gutenprint v5.2 is the bad driver.  Select the correct driver, click Forward, click the radio button to Use the new PPD, and click Apply.  The test page should print correctly now.

Eighth Note: Final Tweak

The keyboard and touchpad on the netbook are pretty tight, and I frequently bump the touchpad while typing.  Of course, this sends my cursor all over the place, and soon I'm typing text three paragraphs above where I should be.  Because I have a mouse, I really don't need the touchpad most of the time, and I want an easy way to turn it on and off.

First, there's a short script. The script stores state information in a status file ($statusFile), and toggles the touchpad on and off.  Be sure to use chmod+x to make the script executable.

#!/bin/bash  

whoAmI=${0##*/}
notifySeconds=5000   
statusFile=/tmp/touchPadOff

# Get the current status.  If there's no status file,
# create one and notify the user.   Otherwise, get the current status
# and set it the opposite way.
touchPadOff=$(cat ${statusFile}  2>/dev/null)
if [  $?  ==  0  ] ;     then 
	if  [  $touchPadOff  ==  0  ] ;      then 
		echo 1 > $statusFile
		synclient TouchPadOff=1
		notify-send -t $notifySeconds  $whoAmI:  'Touch Pad turned off.'
	else
		echo 0 > $statusFile
		synclient TouchPadOff=0
		notify-send -t $notifySeconds $whoAmI:  'Touch Pad turned on.'
	fi
else
	echo 1 > $statusFile
	synclient TouchPadOff=1
	notify-send -t $notifySeconds $whoAmI: 'Touch Pad initialized  and turned off.'
fi
#  

Next use lxhotkey command from a terminal window ( or Preferences -> Setup Hot Keys) to associate a key with the script to execute. Click on the Programs tab, the press the + button to add a new command and hot key. In the pop-up, move the cursor to Hotkey1 and press the desired key.  In my case, I used fn-F9, because that key has a small touchpad image on it.   Lxhotkey identifies this key as XF86TouchPadToggle.  Move the cursor to the command line, enter the fullpath of the script, click on the check mark to save it. 


With this final note, everything is documented, and I'm ready for Lubuntu 20.04 LTS.

Thursday, March 26, 2020

Bebop and Bread

While we're all sheltered in place, it's time for two oldies, but goodies: Bebop and bread.   How do these two go together?

First, bebop:  I just finished reading Ghosts of the Tsunami,  Richard Lloyd Parry's account of Japanese survivors from the 2011 Tsunami.  It's a compelling story of people who lost family and especially children of the Okawa School.   While we're living thru the COVID-19 epidemic,  Ghosts of the Tsumani reminds us that humans have survived terrible hardships before. In one passage,  Parry writes about the Buddhist priest Taio Kaneta and his love of jazz, especially bebop:
"I love jazz", he[Kaneta] said, "and above all I love Thelonius Monk.  Bebop -- such brilliant, peculiar music  It seemed to me that it reflected what people's minds were like after the disaster -- the tempo of people's minds and hearts.  It was the perfect music for the occasion".
So while making a couple loaves of olive bread,  I listened to some bebop, the music of Thelonius Monk.  This is really fabulous music;  it reminds me of my college days, listening to the gravely-voiced DJs on  radio station WBJC proclaiming "Jazz, America's only native art form".

And now for the bread:   This winter hasn't been the best for skiing and snow-shoeing, so one of the things that's kept me busy is bread making.   This morning I made two loaves of olive bread using a recipe from Beth Hensberger's book, Bread.   It's a pretty straightforward recipe; really the only tricky thing is rolling out the dough and rolling in the olive mixture without leaving big air gaps in the final loaf.

After kneading the dough, rolling in the olive mixture, and letting it rise, the bread is ready to bake:

Two loaves of bread waiting to bake

After the bake, sitting on racks cooling, waiting for butter:

Two loaves of bread sitting on cooling racks

Not only is the bread delicious, kneading the dough is very therapeutic for relieving stress, and rising bread teaches patience.  So, knead some bread, wait patiently while it rises, then spoil yourself with warm bread and butter.   It's the perfect antidote for these trying times.




Saturday, July 28, 2018

Simple Web Client, the Sequel

In an earlier post, A Simple Web Client, I demonstrated a simple program for pulling data from a download website to an Oracle table using the htp package.  A few years later, I needed to do a similar data pull to a SQL Server table.  In this post, I'll demonstrate a simple way of pulling data from an URL into a SQL Server table.

In an ideal world, languages and packages would be standard enough that one could readily port a stored procedure from one vendor's database and reuse the code without too much trouble in another vendor's database.    While most SQL is pretty transportable, the stored procedure languages are highly proprietary and do not port at all from one vendor to another ( one exception: IBM's support of Oracle's PL/SQL language).

The  SQL Server solution takes a different approach from the  Oracle PL/SQL solution.   In the latter,  an Oracle PL/SQL procedure used built-in Oracle  packages to open and manage an HTTP connection, to read data from the HTTP socket, and to write the data to the console.  In practice, instead of writing the data to the console, we might have processed the data  directly or loaded the data to a staging table for later processing.   The SQL Server solution is different; rather than use T-SQL to manage the HTTP connection, this solution uses an external program to read the data from the web into a file on a Windows Server before bulk importing the data  into a SQL Server staging table.  After we load the data to a staging table, we can process the data as needed by our application.

Let's look at the SQL Server solution.  First, we'll use the data from the Federal Reserve from the previous post again.  Put the URL https://www.federalreserve.gov/datadownload/Output.aspx?rel=H10&series=122e3bcb627e8e53f1bf72a1a09cfb81&lastObs=10&from=&to=&filetype=csv&label=include&layout=seriescolumn&type=package into your browser.   When your browser opens the URL, you'll get a CSV file like the following:

Downloaded data opened in a spreadsheet

We have a four-column spreadsheet, with some meta-data in rows 1 through 5, the column headers in line 6, and the data starts in row 7.   We'll create a four-column table  to store the results.   In a SQL Server Management Studio (SSMS) window, create the following table:

use MyDatabase
go

create table stageFedRates (
 timePeriod date,
 jrxwtfb_nb dec(15,8),
 jrxwtfn_nb dec(15,8),
 jrxwtfo_nb dec(15,8)
 )
 ;

go

We have a couple of choices of programs to pull data from the web into an external file.  One option is wget, another is curl.   Curl is already installed on my Windows 10 computer and my Windows 2008 Server, so we'll use curl.   The syntax is pretty simple, just type curl and the URL enclosed in double quotes.  Don't forget the double quotes -- the URL is full of special characters that command console will try to interpret if not quoted.  The output goes to STDOUT by default.  Let's try it in a command shell window:

C:\> curl "https://www.federalreserve.gov/datadownload/Output.aspx?rel=H10&series=122e3bcb627e8e53f1bf72a1a09cfb81&lastObs=10&from=&to=&filetype=csv&label=include&layout=seriescolumn&type=package"

"Series Description","Nominal Broad Dollar Index ","Nominal Major Currencies Dollar Index ","Nominal Other Important Trading Partners Dollar Index "
"Unit:","Index:_1973_Mar_100","Index:_1973_Mar_100","Index:_1997_Jan_100"
"Multiplier:","1","1","1"
"Currency:","NA","NA","NA"
"Unique Identifier: ","H10/H10/JRXWTFB_N.B","H10/H10/JRXWTFN_N.B","H10/H10/JRXWTFO_N.B"
"Time Period","JRXWTFB_N.B","JRXWTFN_N.B","JRXWTFO_N.B"
2018-07-09,123.4946,89.6613,160.2890
2018-07-10,123.4637,89.7541,160.0923
2018-07-11,123.8276,89.8616,160.7776
2018-07-12,123.7232,90.1620,160.1318
2018-07-13,124.0710,90.2872,160.7554
2018-07-16,123.8317,90.0232,160.5674
2018-07-17,124.1553,90.4107,160.7806
2018-07-18,124.3573,90.5535,161.0480
2018-07-19,125.0530,90.9072,162.1562
2018-07-20,125.0484,90.9072,162.1456
c:\>

This is the same data we saw in the Excel spreadsheet above. Two useful options of the curl command are the --stderr option and the --output option.   Those two options specify files to store STDERR and the output file:

C:\>curl --stderr curlErrors.txt --output curlData.csv "https://www.federalreserve.gov/datadownload/Output.aspx?rel=H10&series=122e3bcb627e8e53f1bf72a1a09cfb81&lastObs=10&from=&to=&filetype=csv&label=include&layout=seriescolumn&type=package"

C:\>dir curl*.*
 Volume in drive C has no label.
 Volume Serial Number is 1836-25CF

 Directory of C:\

2018-07-28  03:38 PM               804 curlData.csv
2018-07-28  03:38 PM               399 curlErrors.txt
               2 File(s)          1,203 bytes
               0 Dir(s)  352,925,888,512 bytes free

C:\> type curlErrors.txt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   804    0   804    0     0    804      0 --:--:-- --:--:-- --:--:--  2233



Now that we can pull data from our source, let's look at loading into a SQL Server table.  From a new SSMS window,  use the bulk import command to load the table:

use MyDatabase
go

Bulk Insert stageFedRates 
   from  'C:\curlData.csv ' 
   with ( fieldterminator = ',',   
          rowTerminator = '\n',  
          firstRow = 7,
          maxErrors = 999999999 )


The fieldTerminator and rowTerminator options specify that the input is a CSV with rows delimited by new line characters and the columns by commas. We know there is some metadata in the download, so we specify firstRow=7 to start loading data. Finally, maxErrors specifies how many errors the import will tolerate before aborting the load. This will depend on the data; if the data is very clean, specify a small number of errors, but if not, specify a large number so all the data loads into the table for post-load cleanup.

In a SSMS window, select all the data and see what we imported into the SS table:

Data loaded into SQL Server


Now we're able to pull data from the web into a file and load that data into a table. Next, we'll create a stored procedure to automate this process. We've demonstrated all the commands we need to do this, but there's one more hurdle to leap: working within the context of the Windows file system permissions. The directories that we can read/write are not necessarily the directories that SQL Server service can read/write.

To work around this, we will query SQL Server's TMP environmental variable, and we will use that directory to write to and read from. The following stored procedure will do the trick. The comments will explain how it works along way. It's worth leaving the PRINT statements in the code for debugging; getting all the quotes and double-quotes correct can be quite tricky.

use MyDatabase
go

create procedure [dbo].[loadFedReserve] AS
begin
    -- Get SQL Server's tmp directory.    
    declare @destinationDirectory varchar(256);
    declare @cmd varchar(256) = 'echo %TMP%'; 

    -- Create a temporary table 
    create table #tmpResult (
 envVar varchar(256)
 )
    -- Store the results of executing "echo %TMP%" on the Windows server
    -- into the temporary table
    insert into #tmpResult exec xp_cmdshell @cmd 

    -- Now get the TMP directory from the table
    set @destinationDirectory = ( select top 1 envVar from #tmpResult);
 
    -- Build the command line, including the output and stderr file locations.
    -- We will reuse the @destination variable when we import the file to SQL Server.
    -- Be sure to use the double quotes around the URL  
    declare @curlCommand varchar(512) = 'curl ';
    declare @destination varchar(128) =  @destinationDirectory +'\curlData.csv ';
    declare @errors varchar(128) = @destinationDirectory + '\curlErrors.txt" ';
    set @curlCommand = @curlCommand + '--stderr "' + @errors
    set @curlCommand = @curlCommand + ' --output ' + @destination 
    set @curlCommand = @curlCommand + '"https://www.federalreserve.gov/datadownload/Output.aspx?' +  
       'rel=H10&series=122e3bcb627e8e53f1bf72a1a09cfb81&lastObs=10' + 
       '&from=&to=&filetype=csv&label=include&layout=seriescolumn&type=package"'
  
    -- Little bit of debugging, then execute the command
    print  @curlCommand; 
    exec xp_cmdshell @curlCommand
 
    -- The table is usually a staging table that we reuse, so delete any data in it.
    delete from stageFedRates; 

    -- Now build a sql statement with the bulk import command and the file locations
    -- that we determened before
    declare @sqlStmt nvarchar(256) = 'Bulk Insert stageFedRates from  ''' + 
  @destination + ''' with ( fieldterminator = '','', rowTerminator = ''\n'',  firstRow = 7,maxErrors = 999999999 )'
    
    -- A little more debugging, then execute the load.  Building the SQL command and 
    -- executing it with sp_executeSQL let's us change the imported file programmatically. 
    print @sqlStmt;
    exec sp_executeSQL @sqlStmt
 
    select count(*) RowsImported from fedRates
END

GO

Depending on how your SQL Server instance is configured, you may need to enable the use of xp_cmdshell. I can execute the following on the instance of SQL Server on my PC; the help of a DBA is probably required for an enterprise server.

exec sp_configure 'show advanced options', 1;  
go  
-- Refresh the configuration 
reconfigure ;
go 
  
-- Enable xp_cmdshell   
exec sp_configure 'xp_cmdshell', 1;  
go
  
-- Refresh the configuration 
reconfigure ;
go 
  

After compiling the store procedure, open a new SSMS window, execute the stored procedure, and select the results of the load:

exec loadFedReserve

select * 
from stageFedRates
 

There should be 10 rows, like below, and it will take SQL Server less than a second to pull the data and load the table:

Data returned from a SQL Select in SQL Server Management Studio
Using the curl command and a bit of code in a stored procedure, we can pull external data from the web and load it into SQL Server. After loading the data into the staging table,  we can manipulate the data as needed by our business requirements.   The load is fast enough to be useful for interactive processing with small amounts of data, or we can use the SQL Server Scheduler to batch load large amounts of data.



Saturday, April 7, 2018

Lubuntu 17.04 to 17.10 upgrade

Introduction


I'm a bit tardy upgrading my Linux Lubuntu 17.04 distribution.  I have two computers running 17.04, an old HP dv8210us with a single-core, 2GHz AMD processor, and a newer Asus netbook with a dual-core, 1GHz AMD processor.

Everything was working nicely until I tried installing Notepadqq, an open-source version of the popular Notepad++ for Windows computer.  For my next snafu, I tried installing libdvd to play DVDs in my HP.  Unfortunately,  Lubuntu 17.04 is no longer supported, and I couldn't load the pre-requisite packages for either libdvd or Notepaddqq.   Also, security updates are no longer available.

These are my notes and learnings as I worked thru the upgrade process from 17.04 to 17.10.     The notes are my collection of how-tos for the next time I do this.   The learnings are little bits of wisdom I picked up along the way.

First learning:   if you have a choice, install an LTS version of Ubuntu or Lubuntu.  Long Term Support (LTS)  versions are supported for 5 years (Ubuntu) or 3 years (Lubuntu).   This is the second time I've done an upgrade, first from 12.04 to 14.04, and now from 17.04 to 17.10.   The process is not as smooth as one would hope.  My systems are old, things will certainly work better or differently or at least more quickly on your own hardware.

Second learning:  take copious notes of everything you customized and how you customized it.   One problem area on the HP is the Broadcom wireless card.  If you know how to handle it, it's pretty simple.  But the first time thru was painful; following my notes in Google Keep made a messy process easy.

First Note:  Installation media


This is pretty simple,  I created a bootable USB for the netbook (no optical disk player) and a bootable DVD for the HP (no USB boot).  First download the iso image from Lubuntu.net.

To create the bootable USB, use the Startup Disk Creator (System Tools -> Startup Disk Creator). Select the iso image as the source,  a 2GB USB drive as the destination, then click the Make Startup Disk button.

To create the bootable DVD, use xfburn ( Sound and Video > xfburn).   The Burn Image button burns an ISO image to a disc.  A DVD  is required to hold the whole image; a CD is not quite big enough.

Third learning:  procrastination is not your friend.   I upgraded Ubuntu 12.04 to 14.04 using the software center over the network;    four months ago, I could have repeated the process and skipped creating bootable media.

Second Note:  Broadcomm wireless adapter


My old HP laptop has a Broadcomm wireless adapter.  The following notes will help you get this working.   If you don't have a Broadcomm wireless adapter, skip this section.

First, do the install while connected to the Internet with an Ethernet cable. If the system tries to connect to the internet using the existing driver, the system will hang.   You will need to download the correct driver from the internet.   So, plug in.

After installation is complete, remain connected to the Internet via the ethernet cable.  Start a terminal window, and execute the following four commands.
 
$ sudo apt-get update
$ sudo apt-get install firmware-b43-installer
$ sudo apt-get remove bcmwl-kernel-source
$ sudo reboot


Third note:  Consistency


Working on a Windows 10 PC all day at work, I'm accustomed to hitting Windows-L to quickly lock my screen.  I don't have a strong preference for Windows-L on my PC verses Ctrl-Alt-L on my Linux systems, but it's convenient when all the simple things work the same way.

To add this to Lubuntu, edit the lubuntu-rc.xml configuration file in your favorite text editor.  Search for the string "lxsession-default lock".  When you find the string, copy the entire <keybind> thru </keybind>, and paste it below the existing lock command.  Add a comment that the sequence Windows + L locks the screen, and change the "key=C-A-l" to "key=W-l":

  
  <!-- Lock the screen on Windows + l-->
  <keybind key="W-l">
    <action name="Execute">
      <command>lxsession-default lock</command>
    </action>
  </keybind>


Save the file, and from a terminal session refresh your openbox configuration:

openbox --reconfigure

Assuming you didn't get any errors, both Ctrl-Alt-L and Windows-L sequences will lock your screen.

Fourth note:   Two Monitors


I have a second monitor sitting to the left of my laptop screen.   If you know Lubuntu, you know  where this is going...  the left monitor is always the main monitor.  And if you're really "clever", you can swap the monitor positions but find that you don't have a taskbar with the laptop monitor alone.

I've googled this problem, and it's a common question with no satisfactory answer.  After a lot of trial and error, the best solution that I've found is to configure the menu on both monitors identically, and to use xrandr to set the monitors so the second monitor is to the left of the laptop screen.    This gives me a configuration that works whether the laptop has a second monitor or not, and my mouse moves left to go to the left monitor.  With the taskbar on both monitors, I will also have a taskbar on the laptop when only the laptop screen is in use.

Here's scoop:

First, right-click the task bar and open the panel settings.  Configure the panel settings to your personal preferences and close the window.

Next, open a terminal window, and change your working directory to .config/lxpanel/Lubuntu/panels.   Edit the panel file in the panels directory using a text editor like vi or leafpad.  In the global section, add one line

monitor=0

and save the file.   Now change the monitor=0 line to monitor=1, and using a Save as... dialog, save the panel as panel1.

Execute lxpanelctl restart, and you will have taskbars on both of your monitors.

Fifth Note:  Network and Volume Indicators


After installing 17.10, I noticed problems with the network and volume controls on the task bar:  The network menu collapsed so quickly it was almost unusable, and the volume control didn't work at all. After a lot of googling,  I found that this is only a problem if the autohide option on the taskbar is selected.  I like the autohide feature, so I worked around this problem in two ways.   I rarely use the network menu; if I need it, I turn off the autohide feature while I use the network menu.

I use the volume control frequently; so turning off autohide every time I want to mute an advertisement is not ideal.   Instead, I disabled the Sound Menu on the indicator menu and added the Volume Control plug in to the panel menu:
  • Right click on the task bar, and select Panel Settings
  • Click on the Panel Applets tab, select Indicator applets, and click the Preferences button
  • Uncheck the Sound Menu and close the Indicator applets menu
  • Back on the Panel Preferences menu, click the Add button
  • On the Add plugin to panel menu, scroll down the list, find the Volume Control, highlight it, and click the Add button
  • The Volume Control is added to the right of the power control.  Use the Up and Down buttons to move the control where you like.
  • Close the panel preferences

Sixth Note:  More on Network Interfaces


I have two wireless network interfaces with my Asus netbook.  The built-in Qualcomm Atheros AR9485 worked fine out of the box.   In addition, I have a Realtek RTL8192cu USB dongle.   When booting into Windows 10, I found the Realtek interface is much faster than the Qualcomm interface.   In Lubuntu, the Realtek interface would stop working every two or three minutes until I disabled/enabled the interface. Then it would work again for another two or three minutes.

Googling the issue turned up several different answers to the problem; the references below all agree that it's a driver issue.  In short, the driver in kernel 4.4 is supposed to work well, but Lubuntu 17.10 is using kernel 4.13, and the driver for the Realtek interface is buggy.  Here's how to fix it.

sudo apt-get install linux-headers-$(uname -r) build-essential dkms git
git clone https://github.com/pvaret/rtl8192cu-fixes.git
sudo dkms add ./rtl8192cu-fixes
# NB:  The step above reports a version.   
#      As of 2018-04-12, the version is 1.11, not 1.10.
#      Use the correct version in the next step.
sudo dkms install 8192cu/1.11
sudo depmod -a
sudo cp ./rtl8192cu-fixes/blacklist-native-rtl8192.conf /etc/modprobe.d/
sudo cp ./rtl8192cu-fixes/8192cu-disable-power-management.conf /etc/modprobe.d/
reboot


Seventh Note:  Printers


I use two printers at home, a old Lexmark Optra S 1255, and a wireless Epson NX420.  Configuring the Lexmark is easy; it's a postscript printer, and Linux has always supported postscript printers well.   Open System Tools -> Printers -> Add.  I have a network card in the printer, so I expand the Network Printer options and choose AppSocket/HP JetDirect.  Enter the IP of the printer, port 9100 is already filled in, and click Forward.  Select the manufacturer from the list, then the model,  describe the printer, click Apply and print the test page.

The default driver for the Epson NX420 does not work well at all; the colors do not align properly, but instead look like shadows.   This is another problem that's a pain to fix the first time, but it's not too bad the second time.  The NX420 needs a driver from http://www.openprinting.org/printer/Epson/Epson-NX420_Series.  Choose the 64-bit deb driver, download it, and then open the downloaded file.   Lubuntu will open the driver in the Software Center.   Click the Install button to install the driver.

Now add the printer ( System Tools -> Printers -> Add).   Expand the network printers, select the NX420, and click the Forward button.  The system searches for print drivers for a minute or so, then displays the Describe Printer dialog.  Click Apply, and print a test page to verify that the driver works properly.  The test page should identify the driver as EPSON.PPD, and the test page should look perfect.   If the print driver is STP01629.PPD and the images at the top of the test page don't align, then you have the bad print driver.

With these notes, I'm ready for the next upgrade.    With Lubuntu installed on my 12-year old laptop and my under-powered netbook,  I have two pretty responsive computers.   The laptop is perfect for my desk:  if the cat walks across it, who cares? It's a twelve year old computer.  And the one and half pound netbook is great for traveling.


References





Tuesday, January 23, 2018

Snipping in Lubuntu, the sequel

In my last post, I added screen snipping to my Lubuntu system using scrot and mtpaint.   The result is similar to using the Snipping Tool in Windows 7, where the resulting clipped image is left in the Snipping Tool viewer.    There are two drawbacks to the approach:  first, leaving the clipped image in mtpaint is not the quickest thing to do for simple copy and paste applications, and second, not every program will accept the resulting image.

In this post, I'll demonstrate adding a new meta-key combination, Windows-Shift-S to select and copy part of a screen, format the data so any program can use it, and copy the image to the system clipboard.  The end result will be the same meta-key that Windows 10 uses to capture screens, and the result will be ready to paste -- no intermediate step leaving the result in mtpaint.

Capturing the image in a format that was usable by all my programs turned out to be trickier than I thought.   At first, I thought I could send the output of scrot to xclip.   This works for LibreOffice Writer and Gmail, but it didn't work for Blogger.  Hence,it was time to take a deeper dive into this process.  The 'net being the 'net, there's often someone who's run into the same problem.   In this case, artfulrobot (their handle) on askubuntu.com provided a solution that I'm going to work with here.

In this post, I'll discuss using scrot, base64, and xclip to add Snipping Tool functionality to Lubuntu.  We'll start by making sure we have all three programs installed, then we'll test invoking them from the command line, and finally we'll map them to meta-keys in the Openbox window manager.

First, make sure these the tools are installed on your system.   Base64 should be installedd on Lubuntu, but you may need to install scrot and xclip yourself.  Open a terminal window, and try executing xclip and scrot; if they load, everything is good.   If you get a message like:

The program 'xclip' is currently not installed. You can install it by typing: 
sudo apt install xclip 


If you get that message, go ahead and install the tool using apt:

sudo apt install xclip


Now you should have all three tools installed.  There are a lot of moving parts, so let's start with the solution and work backwards.  If you haven't read the previous post, I suggest you browse it now.

First, edit the openbox configuration file using vi or your favorite text editor:

tom@dv8000:~$ cd .config/openbox/ 
tom@dv8000:~/.config/openbox$ ls -l 
total 36 
-rw-r--r-- 1 tom tom 33058 Jan 23 14:06 lubuntu-rc.xml 

tom@dv8000:~/.config/openbox$ cp lubuntu-rc.xml lubuntu-rc.xml.safe
tom@dv8000:~/.config/openbox$ vi lubuntu-rc.xml 


 In our previous post, we added the Windows-s meta-key to copy a screen shot to mtpaint in lines 384 through 389.  The command to do this on line 387 was pretty simple, and we included it right in the configuration file.

The command we need to execute this time is a bit too complicated to include directly in the openbox configuration file, so we'll wrap it in a simple shell script on line 394.  The easiest place to put the shell script is in your home directory until you're satisfied with it.  In my case, that's /home/tom; I'll name the shell script ScrotIt, a meshing of SnipIt and scrot.  Using your editor, add lines 390 through 397, substituting your home directory for /home/tom:


    384     <!--  Windows-s key to select screen and copy to mtpaint -->
    385     <keybind key="W-s">
    386       <action name="Execute">
    387         <command>  scrot -s -e 'mtpaint $f ' /tmp/ScrotSave.png </command>
    388       </action>
    389     </keybind>
    390     <!-- Windows-Shift-s to select screen and copy to xclip -->
    391     <keybind key="W-S-s">
    392       <action name="Execute">
    393         <command>
    394             /home/tom/ScrotIt
    395        </command>
    396      </action>
    397     </keybind>



After you added the new keybinding, save the file and refresh openbox with your changes:

openbox --reconfigure

Next, edit the script ScrotIt using your favorite editor, and add the following lines:

      1 #!/bin/bash
      2 scrot -s /tmp/scrotSave.png && \ 
      3    echo "<img src='data:image/png;base64,"\ 
      4         $(base64 -w0 /tmp/scrotSave.png)\
      5         "' />" | 
      6    xclip -selection clipboard -t text/html 


Let's walk through the code and see what's happening:


  • Line 2 runs the scrot command, saving the output in /tmp/scrotSave
  • Lines 3 thru 5 build an ASCII string with the image
  • Line 3 starts an <img> tag, and the source is a data URI.   The MIME type is image/png, and it will be base64-encoded.
  • Line 4 runs the base64 command, encoding the image data as an ASCII character string.  The bash $( ) construction executes base64 and returns the output as a string to the script running the command
  • Line 5 closes the <img> tag we started in line 3.  The shell concatenates the string in line 3, the output of base64 in line 4, and the closing tag string in line 5 into one long string.
  • Finally, on line 5 the output of the echo command that started on line 3 is piped to xclip on line 6.  xclip takes the output and leaves it in the system clipboard, ready for pasting.

The last step is to make the script executable and test it.  After typing in the ScrotIt command on the second line and pressing enter, ScrotIt will wait until you select an area of your screen.

 

tom@dv8000:~$ chmod +x ScrotIt 
tom@dv8000:~$ ./ScrotIt 


Select an area of your screen and paste it into an application that supports images, like your word processor or mail program (but not a text editor like vi). I copied and pasted the following image into blogger:

Linux terminal screenshot showing Scrotit running

If all went well, now try it with the Windows-Shift-S meta-key combination that you defined for openbox.   After selecting an area of your screen with your mouse, you should be able to paste it into your document.

Using these tools, you can select, copy, and paste from your screen into documents as easily with Linux as you can with Windows 10.



References:

In addition to the Unix man pages for scrot, base64, and xclip, I found the following web pages to be useful.

https://wiki.archlinux.org/index.php/openbox#Keybinds

https://askubuntu.com/questions/759651/how-to-copy-an-image-to-the-clipboard-from-a-file-using-command-line

https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs

Saturday, January 13, 2018

Snipping in Lubuntu

One feature that I find very useful in Windows 10 is the Snipping Tool.   The Snipping Tool is mapped to Win-Shift-s key combination.   After pressing that meta-key combination, the screen grays out, and you can select an area of the screen to copy to the system clipboard.    This is really useful for splatting images quickly into an email (or other document) with a minimum of effort.   Print Screen and Alt-Print Screen were useful for grabbing images of a whole screen or of a whole window.   But many times we only want a small portion of the screen, and the Snipping Tool is a useful way to grab exactly what you need.

In my last post, my old laptop got a second lease on life with a Lubuntu installation and an upgrade to 2GB of memory.  I am spoiled  by the Snipping Tool on my Windows 10 computers, so I'm looking for a way to get that functionality with Lubuntu.

As with many things in life, "da google" is your friend.   Googling "screen capture" or "screen image" returns lots of hits showing screen images from Lubuntu, but not a lot of information describing how to grab that screen image.    Eventually, I found some information about a screen capture utility called scrot (short for SCReen shOT).  In this post, I'll discuss using scrot and mtpaint to add Snipping Tool functionality to Lubuntu.  Mtpaint is installed as part of the Lubuntu distribution, but you may need to install scrot  yourself. Finally we'll map them to meta-keys in the Openbox window manager.

First, make sure scrot and mtpaint are installed on your system.     Open a terminal window, and try executing scrot -v.  If the tool loads and displays a version number, everything is good.   If it doesn't load, you'll see a message like:

The program 'scrot' is currently not installed. You can install it by typing: 
sudo apt install scrot 


If you get that message, go ahead and install the tool using apt:

sudo apt install scrot


Now you should have both tools installed.  I'll start by running them from the command line.  First, try
scrot --select

You'll notice that your terminal window pauses, waiting for you do do something.  With your mouse, click and drag over any section of your screen, creating a rectangle.  Release the mouse button, then in your terminal window type
ls -l *png

and your output will look something like the following.   Scrot creates files with the datetime and images size in the filename:

tom@dv8000:~$ ls -l *png
-rw-rw-r-- 1 tom tom 131103 Jan 13 11:48 2018-01-13-114835_1440x875_scrot.png
-rw-rw-r-- 1 tom tom    510 Jan 13 11:48 2018-01-13-114856_184x172_scrot.png
tom@dv8000:~$


Our next step is to get the the output into mtpaint.  Scrot has a couple of options to facilitate sending the captured image to another program.   The -exec option tells scrot to run another program, and the $f will pass the name of the saved file.   Try the next command; scrot will save the output in /tmp/ScrotSave.png and then execute mtpaint with that filename as the argument.

scrot --select --exec 'mtpaint $f ' /tmp/ScrotSave.png

Now we have all the facilities of mtpaint at our disposal to edit the image.  Using mtpaint, we can save that image back to disk, or just copy it to the system clipboard  using Ctrl-C, then Edit -> Export Clipboard to System from the drop-down menu.

Finally, we'll map this command to a meta-key combination in Lubuntu's openbox window manager.   From your home directory, navigate to the .config directory, then to the openbox directory, and look for the XML configuration file.   In my system, it's lbubuntu-rc.xml:

tom@dv8000:~$ cd .config 
tom@dv8000:~/.config$ cd openbox  
tom@dv8000:~/.config/openbox$ ls -l 
total 36 
-rw-r--r-- 1 tom tom 33422 Jan 13 15:10 lubuntu-rc.xml 
tom@dv8000:~/.config/openbox$ 


There are lots of things that we can configure here, but we're only interested in the keybindings. Before you begin, copy the xml file to a safe place! If you make an error, you don't want to leave the configuration file a mess, and you can restore it using your saved copy.  After creating a backup copy, edit lubuntu-rc.xml with your favorite text editor (leafpad and vi are both installed on Lubuntu, don't use a document editor like Abiword, OpenOffice, or LibreOffice). 

Find the line where the Lubuntu specific keybinding start, and add the XML starting with the <keybind> tag through the closing </keybind> tag:

    382     <!--  Lubuntu specific : Keybindings -->
    383     <!--  Lubuntu specific : Keybindings -->
    384     <!--  Windows-s key to select screen and copy to mtpaint -->
    385     <keybind key="W-s">
    386       <action name="Execute">
    387         <command>  scrot -s -e 'mtpaint $f ' /tmp/ScrotSave.png 
    388      </action>
    389     </keybind>
 

Let's look at what we've done:   Line 384 is a comment describing our change. In line 385, we start a new keybinding, capturing the meta-character combination Windows and the "s" key.  The action will be to Execute the command within the command tags on line 387, which is the scrot command that we executed in the terminal prompt above.

Next, save the file, and refresh the window manager with the new configuration information:

openbox --reconfigure

If all went well, openbox will not display any output.   If there is an error, openbox pops up a message describing the error and its location in the XML file.   After openbox has reloaded the XML file, test your keymap by pressing the Windows and S, then select part of your screen using the mouse.   After you release the mouse key, mtpaint will open with a copy of the image you selected. 

Now you have a quick and easy way to grab a selected part of your screen and manipulate it for further use.  Save the image as a file, or copy it to the system clipboard in mtpaint.




References:

In addition to the Unix man pages for scrot and mtpaint,  the following web pages should be useful for readers desiring a deeper look into the keybinding and mtpaint.

https://wiki.archlinux.org/index.php/openbox#Keybinds

http://mtpaint.sourceforge.net/handbook/en_GB/chap_04.html#SEC6







Saturday, December 30, 2017

He's dead, Jim

Except for Scotty's warning that "the dilithium crystals are going to blow", Dr. McCoy's pronouncement that "He's dead, Jim" may be the most famous Star Trek quote in popular lexicon.

And so was my nine-year old laptop:  it's dead, Tom. With a hard disk that had just failed,  the laptop was "really and truly dead".  Dual-booting into Windows XP or Ubuntu,  the laptop was a top-of-line beast when it was new, and it still performed well enough for most daily tasks nine years later.

A search on eBay and ten dollars later, my laptop was ready to go... except for an operating system.   Restoring Windows XP was not an option; I'm the second owner, and I don't have the XP install disks.   I was running Ubuntu 14.04 LTS at the time the old hard disk failed, so Ubuntu seemed like a good choice this time.

With a single-core AMD 64-bit processor running just shy of 2GHz and  1GB of memory, I noticed that the laptop was finding the latest browsers a handful to run.   This was especially true when Mozilla dropped support for flash video.   HTML 5 video requires much more resources than flash did.  So, I decided to look for a lightweight Linux distribution that required less computing power than the full-blown Ubuntu distribution I was using.  Anything the operating system wasn't using would be available for applications to use.

Google "lightweight Linux distributions", and you'll get dozens and dozens of results to browse through.   My advice:  don't over-think it.   A lightweight distribution is a lightweight distribution, and the 600 pound gorilla is not the Linux distribution you choose (nor is it Windows nor macOS), it's the applications.  Applications are stuffed features that you'll never use, and they get bigger and badder every release.  Browsers like Firefox and Chrome are often victims of massive amounts of JavaScript and advertisements running video automatically.  So, the distribution is the least of your problems, just pick a distribution and install it.

I choose Lubuntu.   It's reasonably lightweight, it has access to the Ubuntu software libraries, it includes enough applications to get you started, and it's well-supported by the Ubuntu community.   I installed Lubuntu 17.04 LTS, which is supported by security updates for the next several years. 

I used Mozilla's Firefox browser until the most recent update that dropped support for flash.  The latest Firefox browser consumed so much memory that the laptop took a deep dive into the swap space and performance really suffered.  I found Opera to work much better.  As a bonus, Opera includes an ad blocker that tames most ads without crippling a website's functionality.

Lubuntu includes Abiword for word processing and gnumeric for a spreadsheet.  They're both lightweight applications, but if you need compatibility with Microsoft's office suite, I recommend installing either LibreOffice or OpenOffice.  LibreOffice and OpenOffice share a common ancestry; either one is a good choice, and compatibility with Microsoft's file types is very good.  Again, don't over-think it! Lubuntu's software catalog includes LibreOffice, so that's the one I choose.     From the Lubuntu start menu, choose System Tools -> Software.   If your computer is old (like mine), be patient while the software catalog loads.   Then, click the Productivity button and scroll down to find LibreOffice.  Installing LibreOffice loads the base product and the word processor; choose Base, Calc, Draw, or Impress to load the other features.

Finally, I found 1GB of memory to be just barely enough to satisfy the memory requirements of the most recent browser upgrades.  If your system has enough memory that it is not using much swap space, you'll find computing to be much more enjoyable.   If your computer's hard disk is buzzing away while it swaps programs and data in and out of memory, it's time for a memory upgrade.   I found 1GB of memory on eBay for $14 that lets my system hum along nicely while barely touching the swap space.    Memory upgrades can be tricky, I had good luck with a-techcomponents, a vendor I found on eBay.

Happy Computing, and Happy New Year!