Tuesday, June 07th, 2011 | Author: bmadsen

A few months ago I converted to full time software engineer away from my past trade as an IT professional serving in various roles.  In this new position, I now work in Eclipse, an IDE, for an average of 8 hours a day.  One thing that has been bugging me has been the annoyance of searching for resources (source files) by name and finding 3 copies of each file referencing the source file (my true interest), the compiled result (which I’m not interested in) and even a couple of copies from linked folders that assist the Eclipse JSP validator in finding resources for it’s code completion and on-the-fly reference checking.

Today it finally bugged me enough that I went digging.  Surprisingly, it only took about 5 minutes to find an attribute on folders named “Derived”.  When set, this flag marks the folder and all resources beneath it as “derived”, or better described as generated or compiled from other source files.

Upon switching this flag for the 3 or 4 locations that contained my duplicate resources in my searches, I found that Eclipse intelligently removed those resource references from my search results because they were marked as “derived”.

Thank you Eclipse for making my life just a little bit easier.  Now, I just need more memory and the ability to manage log4j log files straight from Eclipse and I’ll be set.

Friday, February 25th, 2011 | Author: bmadsen

Introduction

So here’s a bit of wisdom for those interested in setting up FTP-TLS encrypted FTP services.  A very popular ftp server in the Linux world is ‘vsftpd‘.  It is found natively in most Linux distributions these days and is very secure and functional.

One situation I seem to keep running into is the need to securely communicate over FTP services.  There are a few ways to do this:

  1. Encrypt the files prior to transfer with a tool such as PGP
  2. Use a VPN connection and FTP over the VPN
  3. Use an SSH transport FTP such as SFTP
  4. Use FTP encryption, such as FTP-TLS

All of these methods have their place.  However, we will focus on #4 today as it seems to be the lightest weight in terms of administration and ease of use.

The FTP Firewall Fix

One of the ways that FTP (in normal, unencrypted mode) is configured to work over stateful packet inspection firewalls is through the use of the concept of an FTP Helper module.  This module is generally a feature of the firewall software.  It watches the FTP control channel on port 21 for commands issued to open a data connection using PASV mode (otherwise known as Passive Mode).  This means that the firewall configuration need only open TCP port 21 to enable firewall traversal of FTP connections.  Since the firewall can analyze the FTP control channel and dynamically open TCP ports for DATA connections as needed, you no longer have to worry about opening multiple ports to enable FTP communications over common firewalls.

Encryption Using FTP-TLS

Using a server such as “vsftpd” that has the ability to use TLS to allow “explicit” mode encryption through the use of the “AUTH SSL” command, one can enable encrypted communications using FTP style communications.  This feature is then implemented by various popular ftp clients such as FileZilla and Curl, as well as various programming libraries.  One of the major problems that happens, however, when you encrypt the FTP control channel, in addition to the data channel, is that you prevent the FTP Helper feature running on the firewall from being able to read the FTP control commands that request opening of the data connections.  Therefore, the firewall can no longer dynamically open ports to enable data channel connections and your happily functioning FTP server is not able to speak to networks on the other side of the firewall.

To fix this problem, ‘vsftpd’ uses a couple of configuration directives to allow specific configuration of PASV ports to a small range of ports.

pasv_min_port=#####
pasv_max_port=#####

This allows a system administrator to configure only a small range of ports for PASV use (encrypted or not).  This means that, for example, a range of 9000 through 9005 could be configured in ‘vsftpd’ and then specifically allowed through the firewall.  This alleviates the need for the firewall to watch and dynamically open the ports for communications, and your FTP-TLS communications can remain fully encrypted.

Considerations

Something to consider is that in picking the particular ports you bind to the ftp server for passive communications, you should consider a system’s configured “Ephemeral Ports“, or the ports that local clients use to open outbound connections.  You should also take care to put the range in an unused range where no system services are already using the ports.

Friday, February 18th, 2011 | Author: bmadsen

I’ve been working with Subversion a lot lately and had to revert a file deletion that occurred a year ago in my employer’s code-base.  Doing a quick Google search, I found this article:

http://www.canfield.com/content/svn-restore-deleted-file

The article got me on the right track, but I found that I could drop the “-r (revision)” parameter and just do this:

svn copy https://{URL}/canfield.conf@98 .
svn commit

This keeps the history of the file during the restore process, which some of the other methods may not do.  One important note is that you go back to the revision before the file was deleted.

Wednesday, June 02nd, 2010 | Author: bmadsen

I happened to read a RWW article on Tweeting in Color today.  I can’t say I was thrilled at the entire concept of Twitter Style Sheets (TSS), the Twitter version of CSS for HTML.  The entire concept of having people able to colorize and change the font characteristics of their tweets is a bit disturbing to me.  To quote the last section of the article:

For now, come check out our demo (if you have a preview access account), or at least read the code (at http://xent.com/tss.js ) and stay tuned for what comes next. Though, like @ev himself, we’re not quite sure about the Promethean curse of giving users Tweets in any color they’d like – we’re afraid it’s only a matter of time before we see animated, blinking Tweets in 3-D!

I can’t say I’m hopeful at the ability of at least some of Twitter user base to restrain from posting loud and obnoxious tweets, or louder and more obnoxious, anyway.  What I do like about that post is the mention of annotations and that people are thinking about how to link between tweet subjects and other social network accounts using the annotation functions.  It is indeed difficult to reference a Facebook user in a Tweet in a useful manner while keeping the message under 140 chars.

What interested me more about the article was the thoughts that it prompted about the usefulness of the Internet.  The concepts of “Content is King” and “keeping presentation out of the way of your message” seem to come to mind when being reminded of the old “blink” HTML tag and it’s possible use (or abuse) in the future of Twitter.  It is a very good reminder that new functionality can be misused just as easily as it can be used to improve experience.  That’s something every UI designer and software engineer needs to be reminded of on a regular basis.

Wednesday, June 02nd, 2010 | Author: bmadsen

I was forwarded an article today that talked about the benefits of SSDs (solid-state-disks) on the performance of MySQL databases.  It sounds both amazing and almost unbelievable.  There has to be a catch, right?

Well, there probably is, but I’d love to see the technical scenarios behind which the 4 companies referenced in the article claim that much of a benefit.  None of my projects are yet to the point of scale where it would be likely, but I hope they will be soon.  I could probably learn a LOT from setting up informal interviews with the engineers behind those architectures.

Then I got to thinking about MySQL database server architecture and some consulting I’ve been doing for a friend of mine.  He’s running a fairly popular web forum based on raising chickens in your backyard.  Go ahead and pop over to www.backyardchickens.com if you’re interested in the topic.  He’s been facing some load problems with his database and his server environment he’s using.  He has some architectural decisions to make soon that have been heavily weighing on him lately, not the least of which deal with the size and structure of his MySQL database.

My thinking in his scenario where his load has gotten high enough to warrant fairly advanced performance tuning of his database, leaned me to thinking about one tunes a MySQL database, specifically when running the InnoDB or in-memory table engines.  Basically, the more memory you can throw at the system (and assign to MySQL for in-memory caching), the better it’s going to perform.  (Tuning gets much more advanced than that, but that’s the main portion of it).  It’s imperative these days that you use a 64-bit processor and matching OS and software if you have any sort of size on your database, lest you run into the 2GB process memory limit of 32-bit systems.

However, the reason that caching mechanism works so well is that it reduces the number of reads from disk that need to be executed.  It stands to reason, thinking logically anyway, that if you can get the read to write ratio down far enough, the disk subsystem will be spending most of it’s time writing.  If SSDs are supposed to perform much better than their standard disk counterparts, than the little time MySQL spends reading from the disks would be even faster, and the write performance would be even less, leaving much more time to spend answering queries from memory cache.

Of course, this is all hypothesis on my part since I am not deeply skilled in the arts of MySQL performance tuning.  Nor do I know a significant amount about SSDs, definitely not enough to say for certain whether their performance attributes would definitely be of benefit to specific MySQL architectures and design use cases.

Wednesday, May 26th, 2010 | Author: bmadsen

In the last few years of my professional career, I have had the opportunity to work in various programming languages, operating system environments and team dynamics.  I have also recently been given the task of deciding what type of products I want to be designing and developing.  What I’m about to write were thoughts specifically triggered as a result of a conversation I had today with my brother, who is a very intelligent creative design type.

He has been talking up the idea of creating various Wordpress themes and plugins as well as various iPhone applications, mostly as a way to get his (or our) names out there as knowledgeable people in the respective fields of programming.  This would ideally lead to a consulting agency for project-based work for larger clients wanting more specialized development tasks performed.  It sounds like a very good plan, to be honest, if you’re into that kind of work experience.  Sometimes I think I am, but then most times I feel a bit more grounded than that…

It takes a talent for multitasking to be in the consulting agency environment.  I’ve tried it out a few times, and I can honestly say that I find the various unrelated projects from differing customers to be distracting to one another, and to the flow of my creativity.  Now, this experience has been in the IT consulting field, which may be entirely different from the programming consulting field, but I suspect the effect is simply less intense and that the short term nature of consulting arrangements tends to distract from the longer-term nature of designing robust software applications.

Much more interesting to me, personally, is to take a project from its concept, architect it well, and help it grow into a fully mature and stable product that somebody finds useful.  The pressure is a little less intense as you can feel free to make mistakes with design, as you will have (more) ownership, responsibility and the inherent ability to fix those mistakes given the longer timeline of feedback from users.

To all my future readers, regardless of the programming slant to this post, what are your thoughts on consulting versus long-term projects?

Friday, March 13th, 2009 | Author: bmadsen

I was catching up on my blog reading tonight and found a really interesting blog post over at Writer Dad.  The very first article I saw caught my attention as I’ve identified myself with the ability referred to in a comment on that article by Randi: “Education isn’t about knowing all the answers. It’s about knowing where to FIND the answers when you have a question.”  Randi then referred to the Socratic method of teaching, which I’m putting on my list of things to read up on.

I guess I learn something new every day.  Time to start asking more questions ;-)

Sunday, January 11th, 2009 | Author: bmadsen

I’ve been reading an interesting book by the name of “A Whack on the Side of the Head” over the last many months.  One of the interesting concepts that I’ve tried to leave myself open to is that of “Cross-fertilization”.  This would be the theory that you can learn many concepts from areas of expertise outside of your own.  A software engineer can learn things from a gardener.  A professional sports player can learn things from a grade school teacher.  Well, I have been involved with my wife in team teaching 3 year old children in Sunday School at my church.

Now, I must give a lot of credit to my wife who is an excellent teacher for children.  She is absolutely fabulous at grabbing the attention of children and teaching them at the level they’re ready to learn.  She loves to do it to.  And to watch and participate in that instruction and activity is inspiring on so many levels.

What occurred to me today, however, was that ties can me made to the concept of knowing your audience, and communicating to them only what they are prepared and interested in hearing and learning.  So much of the time in marketing campaigns this concept is largely underestimated.  Many campaigns focuse simply on age range and/or gender for consumers or company size and revenues for businesses.  However, most cold calling and mailer campaigns (not to mention email campaigns) I’ve heard of usually have many more people simply ignore them than actually respond or end up converting into a sale.

However, it’s not just the marketing department that has a tough time at truly understanding their audience.  In fact, they probably understand that concept quite well given that they deal with it in such an intimate fashion.  Many other business units suffer from the problem of being unaware of their audience’s needs and interests in and how to communicate with them about them regarding those topics in a manner that both parties understand.  For example, how many times does an IT department truly understand where the executive team wants going with the business.  What if the IT departments took the time to learn how to communicate ideas and capabilities of technology to executive teams that were business savvy rather than tech-savvy?  What if engineering teams took the time to work with materials acquisition experts to understand the costs of their designs?  (to be honest, I’m not sure the good ones don’t)

My point is that many benefits of efficiency would be realized.  Costs would be reduced, businesses would be more nimble, marketing campaings would be more effective and less annoying.  These are just some examples.  I could probably do better at explaining HOW to find out more about your audience, but I’ll save that for another post.  Something about suppliers and consumers being willing to engage in open conversation about their products and services and consumers valuation of them…  Yeah, that’s a topic that I think I need to explore more about.

Saturday, November 29th, 2008 | Author: bmadsen

Given that I had the opportunity to travel this Thanksgiving week to my friend’s place in Los Angeles, I had a lot of time to think (and a lot of time to regret the decision to drive with my 3 little boys on a holiday weekend, but that’s another post in another blog ;-) ).  Anyway, as a result I have the following short list of things I am thankful for in things tech related.

  • Engineering geniuses that understand that they don’t know everything and can find bits of truth anywhere
  • Engineering humility that allows cooperation with designer genius
  • Palm, for leading the way in smart phones with their PDA series so long ago
  • Apple’s iPhone, for forcing the mobile phone industry to go ’sexy’
  • the Internet and it’s creators, for enabling world-wide connection, information sharing and collaboration
  • Google, for enabling the search of and usage so much useful and relevant information
  • Google watchdogs for reminding us that so much power can also be easily abused
  • Facebook, for allowing me to connect with so many of my old friends
  • Twitter, for introducing microblogging and the “Continuous Partial Attention” concept
  • Microsoft marketing, for making computers look easy to end users and their developer tools group for enabling such sophisticated development
  • Many other companies for creating technology that actually makes computing easy

In all seriousness, there are plenty of other things I am thankful for related to tech.  I am more thankful to my family and friends for putting up with my incessant use of “Greek” language that tends to put them to sleep ;-)

Monday, November 17th, 2008 | Author: bmadsen

It pains me that I have anybody on my list of Facebook friends that would even consider this chain-letter for a minute…  Today I (along with 14 of my other “closest” friends) received the following message:

Attention all Facebook users
Facebook is recently becoming very overpopulated,
There have been many members complaining that Facebook
is becoming very slow.Record shows that the reason is
that there are too many non-active Facebook members
And on the other side too many new Facebook members.
We will be sending this messages around to see if the
Members are active or not,If you’re active please send
to 15 other users using Copy+Paste to show that you are active
Those who do not send this message within 2 weeks,
The user will be deleted without hesitation to create more space,
If Facebook is still overpopulated we kindly ask for donations but until then send this message to all your friends and make sure you send
message to show me that your active and not deleted.

Founder of Facebook
Mark Zuckerb

My reaction:  Ugh…  Do people really take this kind of thing seriously any more?  I mean seriously…  Reading the email and applying the smallest amount of commen sense would say that sending MORE messages would only serve to further burden the system, a bad idea if it were really having performance problems in the first place.

What it DID remind me of, however, is the willingness of people to believe anything without understanding where to go in order to “check the facts.”  This is a problem that the security industry is forever battling with end-users that are not very technically or security saavy.  I guess we will always still have work to do in educating our friends and neighbors.