Archive for the Category » TechTalk «

Tuesday, June 07th, 2011 | Author: bmadsen

A few months ago I converted to full time software engineer away from my past trade as an IT professional serving in various roles.  In this new position, I now work in Eclipse, an IDE, for an average of 8 hours a day.  One thing that has been bugging me has been the annoyance of searching for resources (source files) by name and finding 3 copies of each file referencing the source file (my true interest), the compiled result (which I’m not interested in) and even a couple of copies from linked folders that assist the Eclipse JSP validator in finding resources for it’s code completion and on-the-fly reference checking.

Today it finally bugged me enough that I went digging.  Surprisingly, it only took about 5 minutes to find an attribute on folders named “Derived”.  When set, this flag marks the folder and all resources beneath it as “derived”, or better described as generated or compiled from other source files.

Upon switching this flag for the 3 or 4 locations that contained my duplicate resources in my searches, I found that Eclipse intelligently removed those resource references from my search results because they were marked as “derived”.

Thank you Eclipse for making my life just a little bit easier.  Now, I just need more memory and the ability to manage log4j log files straight from Eclipse and I’ll be set.

Friday, February 25th, 2011 | Author: bmadsen

Introduction

So here’s a bit of wisdom for those interested in setting up FTP-TLS encrypted FTP services.  A very popular ftp server in the Linux world is ‘vsftpd‘.  It is found natively in most Linux distributions these days and is very secure and functional.

One situation I seem to keep running into is the need to securely communicate over FTP services.  There are a few ways to do this:

  1. Encrypt the files prior to transfer with a tool such as PGP
  2. Use a VPN connection and FTP over the VPN
  3. Use an SSH transport FTP such as SFTP
  4. Use FTP encryption, such as FTP-TLS

All of these methods have their place.  However, we will focus on #4 today as it seems to be the lightest weight in terms of administration and ease of use.

The FTP Firewall Fix

One of the ways that FTP (in normal, unencrypted mode) is configured to work over stateful packet inspection firewalls is through the use of the concept of an FTP Helper module.  This module is generally a feature of the firewall software.  It watches the FTP control channel on port 21 for commands issued to open a data connection using PASV mode (otherwise known as Passive Mode).  This means that the firewall configuration need only open TCP port 21 to enable firewall traversal of FTP connections.  Since the firewall can analyze the FTP control channel and dynamically open TCP ports for DATA connections as needed, you no longer have to worry about opening multiple ports to enable FTP communications over common firewalls.

Encryption Using FTP-TLS

Using a server such as “vsftpd” that has the ability to use TLS to allow “explicit” mode encryption through the use of the “AUTH SSL” command, one can enable encrypted communications using FTP style communications.  This feature is then implemented by various popular ftp clients such as FileZilla and Curl, as well as various programming libraries.  One of the major problems that happens, however, when you encrypt the FTP control channel, in addition to the data channel, is that you prevent the FTP Helper feature running on the firewall from being able to read the FTP control commands that request opening of the data connections.  Therefore, the firewall can no longer dynamically open ports to enable data channel connections and your happily functioning FTP server is not able to speak to networks on the other side of the firewall.

To fix this problem, ‘vsftpd’ uses a couple of configuration directives to allow specific configuration of PASV ports to a small range of ports.

pasv_min_port=#####
pasv_max_port=#####

This allows a system administrator to configure only a small range of ports for PASV use (encrypted or not).  This means that, for example, a range of 9000 through 9005 could be configured in ‘vsftpd’ and then specifically allowed through the firewall.  This alleviates the need for the firewall to watch and dynamically open the ports for communications, and your FTP-TLS communications can remain fully encrypted.

Considerations

Something to consider is that in picking the particular ports you bind to the ftp server for passive communications, you should consider a system’s configured “Ephemeral Ports“, or the ports that local clients use to open outbound connections.  You should also take care to put the range in an unused range where no system services are already using the ports.

Friday, February 18th, 2011 | Author: bmadsen

I’ve been working with Subversion a lot lately and had to revert a file deletion that occurred a year ago in my employer’s code-base.  Doing a quick Google search, I found this article:

http://www.canfield.com/content/svn-restore-deleted-file

The article got me on the right track, but I found that I could drop the “-r (revision)” parameter and just do this:

svn copy https://{URL}/canfield.conf@98 .
svn commit

This keeps the history of the file during the restore process, which some of the other methods may not do.  One important note is that you go back to the revision before the file was deleted.

Wednesday, June 02nd, 2010 | Author: bmadsen

I happened to read a RWW article on Tweeting in Color today.  I can’t say I was thrilled at the entire concept of Twitter Style Sheets (TSS), the Twitter version of CSS for HTML.  The entire concept of having people able to colorize and change the font characteristics of their tweets is a bit disturbing to me.  To quote the last section of the article:

For now, come check out our demo (if you have a preview access account), or at least read the code (at http://xent.com/tss.js ) and stay tuned for what comes next. Though, like @ev himself, we’re not quite sure about the Promethean curse of giving users Tweets in any color they’d like – we’re afraid it’s only a matter of time before we see animated, blinking Tweets in 3-D!

I can’t say I’m hopeful at the ability of at least some of Twitter user base to restrain from posting loud and obnoxious tweets, or louder and more obnoxious, anyway.  What I do like about that post is the mention of annotations and that people are thinking about how to link between tweet subjects and other social network accounts using the annotation functions.  It is indeed difficult to reference a Facebook user in a Tweet in a useful manner while keeping the message under 140 chars.

What interested me more about the article was the thoughts that it prompted about the usefulness of the Internet.  The concepts of “Content is King” and “keeping presentation out of the way of your message” seem to come to mind when being reminded of the old “blink” HTML tag and it’s possible use (or abuse) in the future of Twitter.  It is a very good reminder that new functionality can be misused just as easily as it can be used to improve experience.  That’s something every UI designer and software engineer needs to be reminded of on a regular basis.

Wednesday, June 02nd, 2010 | Author: bmadsen

I was forwarded an article today that talked about the benefits of SSDs (solid-state-disks) on the performance of MySQL databases.  It sounds both amazing and almost unbelievable.  There has to be a catch, right?

Well, there probably is, but I’d love to see the technical scenarios behind which the 4 companies referenced in the article claim that much of a benefit.  None of my projects are yet to the point of scale where it would be likely, but I hope they will be soon.  I could probably learn a LOT from setting up informal interviews with the engineers behind those architectures.

Then I got to thinking about MySQL database server architecture and some consulting I’ve been doing for a friend of mine.  He’s running a fairly popular web forum based on raising chickens in your backyard.  Go ahead and pop over to www.backyardchickens.com if you’re interested in the topic.  He’s been facing some load problems with his database and his server environment he’s using.  He has some architectural decisions to make soon that have been heavily weighing on him lately, not the least of which deal with the size and structure of his MySQL database.

My thinking in his scenario where his load has gotten high enough to warrant fairly advanced performance tuning of his database, leaned me to thinking about one tunes a MySQL database, specifically when running the InnoDB or in-memory table engines.  Basically, the more memory you can throw at the system (and assign to MySQL for in-memory caching), the better it’s going to perform.  (Tuning gets much more advanced than that, but that’s the main portion of it).  It’s imperative these days that you use a 64-bit processor and matching OS and software if you have any sort of size on your database, lest you run into the 2GB process memory limit of 32-bit systems.

However, the reason that caching mechanism works so well is that it reduces the number of reads from disk that need to be executed.  It stands to reason, thinking logically anyway, that if you can get the read to write ratio down far enough, the disk subsystem will be spending most of it’s time writing.  If SSDs are supposed to perform much better than their standard disk counterparts, than the little time MySQL spends reading from the disks would be even faster, and the write performance would be even less, leaving much more time to spend answering queries from memory cache.

Of course, this is all hypothesis on my part since I am not deeply skilled in the arts of MySQL performance tuning.  Nor do I know a significant amount about SSDs, definitely not enough to say for certain whether their performance attributes would definitely be of benefit to specific MySQL architectures and design use cases.

Wednesday, May 26th, 2010 | Author: bmadsen

In the last few years of my professional career, I have had the opportunity to work in various programming languages, operating system environments and team dynamics.  I have also recently been given the task of deciding what type of products I want to be designing and developing.  What I’m about to write were thoughts specifically triggered as a result of a conversation I had today with my brother, who is a very intelligent creative design type.

He has been talking up the idea of creating various Wordpress themes and plugins as well as various iPhone applications, mostly as a way to get his (or our) names out there as knowledgeable people in the respective fields of programming.  This would ideally lead to a consulting agency for project-based work for larger clients wanting more specialized development tasks performed.  It sounds like a very good plan, to be honest, if you’re into that kind of work experience.  Sometimes I think I am, but then most times I feel a bit more grounded than that…

It takes a talent for multitasking to be in the consulting agency environment.  I’ve tried it out a few times, and I can honestly say that I find the various unrelated projects from differing customers to be distracting to one another, and to the flow of my creativity.  Now, this experience has been in the IT consulting field, which may be entirely different from the programming consulting field, but I suspect the effect is simply less intense and that the short term nature of consulting arrangements tends to distract from the longer-term nature of designing robust software applications.

Much more interesting to me, personally, is to take a project from its concept, architect it well, and help it grow into a fully mature and stable product that somebody finds useful.  The pressure is a little less intense as you can feel free to make mistakes with design, as you will have (more) ownership, responsibility and the inherent ability to fix those mistakes given the longer timeline of feedback from users.

To all my future readers, regardless of the programming slant to this post, what are your thoughts on consulting versus long-term projects?

Sunday, November 09th, 2008 | Author: bmadsen

I just ran across a post today that recommends people stop blogging and move on to other technology to communicate.  The article, linked here, comments that most blogs are relegated to the dark corners of the blogosphere because they just don’t have the traffic to get noticed anymore.  Also, what is mentioned is that those that do get comments, get hecklers more than anything.

To this I would say: “Bah”, you’re missing the point of blogging for a great number of people that actually use the technology these days.  The point about microblogging, countered appropriately by many in the comments, is invalid since most bloggers are out to either record their experiences for Google referencing on very specific topics, or are out to disseminate information to a very targeted crowd such as family or a close network of friends and colleagues anyway.

Furthering this point of view is the idea that blogs are no longer primarly “publicized” by means of Google or aggregation sites.  They are publicized by means of other bloggers, social network utilities and content aggregation sites like FriendFeed or MyBlogLog.

So, as I continue my exploration of blogging and it’s function in my life, I continue to see a lot of use for it.  I can’t imagine, for example, how I would post these thoughts in a Twitter style microblog.  Nor could I see creating the entries I put in my various blogs into the limited “notes/blogging” services of MySpace or Facebook.

Sunday, November 09th, 2008 | Author: bmadsen

Ah, I love Wordpress and it’s use of Ajax for it’s “Auto-Save Draft” feature.  I had put a bit of thought into a post last night, then I shut my computer off before thinking to finish the post and publish it.  I was grateful this evening when I logged back in, remembering I hadn’t posted the message, and the draft was sitting there, quite current, waiting for me to finish my thoughts and publish it.  More proof that I love this platform ;-)

Wednesday, October 15th, 2008 | Author: admin

Ok, so WordPress finally and completely won me over today.  They have a wealth of plugins, themes, and a huge community supporting their ecosystem.  Whether you’re running your own web-blog, a photo-blog or even a podcast, there are plugins to support that.  Since it’s free, it’s even a great resource for small and large enterprises alike.  I’m hosting my own instance of WordPress-MU simply to make it easier to maintain the 2 or 3 core blogs of different topics I am interested in, and the soon to be 2 or 3 more “test” blogs I’ll be creating as I attempt various less-common blog types such as photo-blogging and knowledge-blogging (a probably mistaken name for the concept of blogging at least one bit of information that you learned about on any given day).

I knew I’d do it too…  This is actually an edit since I posted the entry and then remembered the big feature that I just learned of that made me think to write this post.  There are instructions on importing your blog from wherever you hosted it last, and the instructions are quite clear.  I have found that this is very compatible with the overall experience of the latest versions that I have seen of WP.  Things are laid out simply, clearly and the user experience seems to make a lot of sense.

Tuesday, June 26th, 2007 | Author: bmadsen

A colleague of mine recently brought up the idea of “What if it don’t?”
As in, what if it doesn’t do what is expected? Particularly, he was
speaking of programming and running method calls. In his words “most
programs handle the ‘what if it does’ exactly what it says the routine
will do. However, the well written ones handle the ‘what if it don’t’
do exactly what it says it will do.” His thought was that the well
written applications handle exceptions cleanly and elegantly rather
than spitting some random error message out to the end user who has no
power to control why the error message occurred in the first place.

A recent article on ZDNet entitled Google�s three rules
emphasized a software and overall data center design philosophy of
expecting things to break and writing error handling routines into
their environment. It shocked me to read the responses and to see the
prevailing mindset of many applications developers in this day and age.
It shocked me even more to see that mindset prevalent in the enterprise
software space, though I suppose it shouldn’t have given the number of
stories people hear of enterprise software being rigid, inflexible and
generally very difficult to work with.

What rings true in the design is that more effort was placed on
handling the error conditions that are going to exist no matter what
hardware you throw at a problem, rather than throwing complicated
hardware designs and configurations at a problem that needs to be
solved by software anyway. Hardware will fail. Even software will fail.
It doesn’t matter how much effort you throw at it to try to get it to
not fail, it will always find a way. The elegance of an engineer’s
design is in how it handles failures and recovers from them.