Doc's Thoughts

Plans are for wussies

2007-01-28T09:20:00.000-05:00

Several months ago, I posted some statements I heard from "professionals" regarding testing, standards, and code review. Since then, I've heard several more gems and here are a few on planning that I thought worth noting.

These statements are from the same person on the same day pertaining to the same project:

"We have a plan. The problem is not a lack of plan. The problem is our communication."

"We don't need a written plan for this (release of a new product). We are experienced professionals and we know what to do. Writing out a plan is to make other people happy; people who don't have to do the work and don't know how to do it."

"You can't plan for something like this. Things come up and you just take care of them."

So let me see if I got this one right.... You have a plan for the thing that can't be planned, but since you are professionals you don't need to communicate the plan for the thing that can't be planned because only ignorant observers need plans? Is that it?

Let me share with you some common challenges within this environment:

Team members have differing expectations of what is to be done, by whom, and when
Team members have no idea what other people on their team are working on
Almost all projects are late due to unforeseen circumstances such as the code that was written won't run in the target environment (actually happened)
Projects get delayed because other departments are "uncooperative "

Other departments don't know what is going on
Other departments want justification for the tasks they are told to do
Other departments require more heads up on simple tasks like configuring a VLAN and building all the servers to the unwritten specification that is needed immediately
Other departments won't violate security standards or other documented practices in order to "get it done"

Once a project "feels" like it is about half-way complete, it is "almost done". Sometimes projects are "almost done" two or three times as long as any other project phase.
Completed work sits on the stage server for months or even years because nobody tests it and certifies it to go to live.

Final note - lack of planning != Agile

Getting Some Help

2007-01-14T14:28:00.000-05:00

I'm not talking about seeing a phychologist or psychiatrist. I'm not talking about a 12-step program. And I'm certainly not talking about booking a stint on Dr. Phil. I'm talking about reaching outo to your team for input, for feedback, and for another pair of hands. I'm talking about actual teamwork.

Developers who struggle with the same piece of code, won't leave their cube, won't acknowledge a problem, and refuse to get anyone else involved. They sometimes become paniced or desparate. They stay extra hours alone, suffering under their self-assigned yolk, blaming others for what is happening.

Managers who can't deliver a project on time and whose staff is unmotivated. Half of the staff know nothing about the project, tasks are not getting done, and new unforseen issues keep popping up. But the manager will tell you they're doing everything they can while they refuse anyone's help. They too become paniced or desparate, putting in heroic effort when it is all but too late, suffering under their self-assigned yolk, blaming others for what is happening.

I've seen this all too often. I've been these people.

As a manager, it is not your job to know every last detail of the project. As a manager, it is not your job to assign every task. You are not supposed to have all the answers or even understand every part of the solution. As a manager, it is your job to enable the team, solicit their input, guide them, direct them, clear their obsticles, and let them do their jobs well. This doesn't mean you aren't in control. You set the standards, you set the expectations, you monitor what is happening, you address the issues, and you participate without dictating.

So why did I call this post "Getting Some Help"?

Because the help you need comes from your team. As a team member or as a manager, unless you are literally a team of one, help is always available. Ask your fellow programmer to take a look at the code that haunts you. Ask your team to review the plan or ask your team to help put together the plan.

What does it mean to be Agile?

2007-01-11T22:01:00.000-05:00

What is Agile?

Is it XP, SCRUM, or some other process? If you are using all of the SCRUM mechanisms, are you Agile?

Agile is a philosophy, NOT a set of practices. This is not to say that those that use SCRUM mechanisms are not Agile. It is only to say that implementation of mechanisms does not make you Agile. And most certainly, failure to implement certain mechanisms does not mean you are not Agile.

I've heard on several occassions statements such as "Oh, you don't do pair programming? Then you're not doing Agile." Pair programming could be replaced with standing meetings, weekly iterations, monthly iterations, test driven development, or any other mechanism.

I intend to discuss more about Agile in this Blog. What do I think it is and what have I learned about it.

Statements I have recently heard

2006-08-10T06:42:00.000-04:00

I had an conversation recently with a number of folks; all of whom are having fairly consistent problems with their system's stability. Each of these statements come from individuals who make their living as a senior level developer or a manager of a team of developers.

Can you tell them why they are having problems?

These statements are from the same person:
"We don't have the time to test thoroughly and meet out deadlines. Besides the users are supposed to test."
"We test the application every which way from Sunday. Testing wouldn't have caught the [run-time] error."

As a note - the error was fundamental and a standard regression test of the module explicitly changed would have caught it (in my opinion).

Other great statements:
"Standards are just loose guidelines. As long as I can read the code and it works, it's good. I don't need to be picky." - responsible for team of 7 developers working on the same code base. The week prior to this statement, the system had crashed due to use of poorly scoped variables (something you often see covered in standards)

"Patterns (such as the GoF patterns) are a great theory, but we code in the real world." - HUH?

"We like to make fun of [insert name here]. He's a real stickler for standards and stuff." - responsible for team of 20+ developers all working on the same application - making reference to one of their own employees.

"We almost had to quit doing code reviews... people kept criticising each other's code." - one of my personal favorites

Code Naming Conventions

2006-08-07T16:37:00.000-04:00

There is a pretty good (albeit brief) series on Code Naming Conventions by The Igloo Coder:
http://igloocoder.com/archive/2006/07/04/394.aspx

The series first looks at the need for conventions and essentially makes the argument that consistency is paramount to good team coding. I agree.

He then covers Local Variable, Module Variables, Functions, Methods, Parameters, Properties, Classes, Interfaces, Super (and Base) Classes, UI Controls, and SQL Server Objects (also applicable to Oracle, MySQL, etc.)

I happen to agree with his observations as well as his conclusions. These are the same standards I use when something else is not mandated.

The following is a brief summary. Do read the series; there is more detail. It's an easy read.

Local Variables	strHungarian (Yes, seriously, Hungarian Notation) strEmployeeName
Module Variables	_strHungarian _intRecordCount
Functions	PascalCase (Verb-Noun) DeterminePrice()
Methods	PascalCase (Verb-Noun) PoundSalt()
Parameters	camelCase DoThing(bool isGoodBoy) * He actually makes note to avoid bool and use enum instead
Properties	PascalCase read his blurb for more details
Classes	PascalCase
Interfaces	IPascalCase ICanPontificate
Super(Base) Classes	Read his blurb
UI Controls	strHungarian txtUserName * Alternate is to prefix HTML controls with 'h' => htxtUserName
SQL Server Objects	PascalCase (Noun-Verb) EmployeeAdd * The Noun-Verb is specific to Stored Procedures

JSR 168, WSRP, Portlets & Enterprise Portal

2006-08-03T11:34:00.000-04:00

Decent Blog on the general Portal Market, the emerging standards, and the players:

JSR 168, WSRP, Portlets & Enterprise Portal

Heartbreaker

2006-07-31T20:31:00.000-04:00

My wife and I have decided to sell off one of our small side businesses. The effort is too great with our full time careers. This site would make a nice part time job for somebody. The potential is far greater for the right person.

Candles and Accessories Web Site for Sale

http://www.sitepoint.com/marketplace/auction/642

Good Article

2006-07-31T20:05:00.000-04:00

I am literally weeks behind on my Blog. I have nearly one dozen items I am working on that I have not posted yet. But today I ran across an article I had tagged and almost neglected to read.

I was impressed with this article. Not only because it bestows the values I believe an organization should genuinely hold and suport, but because it comes from one with a very impressive track record. A man, who for all practical purpose, had no reason to doubt his ways.

http://www.fastcompany.com/magazine/02/meyerson.html

AD World - Day 2

2006-07-18T20:13:00.000-04:00

Update on Unified Process
"Idea that each team should follow the same process is a great theory, but is impossible in practice. Goal should be to have all teams using similar processes." – presenter’s opinion
"Firehose" Presentation – lots of data fast
Speaker has done the gambit on project management – strict to Agile. Has drafted industry standards.
First public release of RUP was June 1998. Slow activity through late 90s. Much work on application of Agile w/in RUP over past six months.
http://www.enterpriseunifiedprocess.com/essays/history.html
UP is a framework from which system processes are instantiated

Spirit of the UP
· Attack risks early (or they will attack you)
· Deliver value to the customer
· Focus on executable software
· Accommodate change early in the project
· Baseline an executable architecture early on
· Build with components
· Work together as one team
· Communicate early and often
· Make quality a way of life
There will be a new version of RUP released later this year

RUP Universal Principles
· Adapt the process
· Balance stakeholder priorities
· Collaborate across teams
· Demonstrate value interactively
· Elevate the level of abstraction
· Focus continuously on Quality

LOOK AT AGILE 2.0
Working with Legacy Code – book on evolving legacy systems over time
Refactoring Databases – book on improving legacy database schemas
RUP original six best practices are now encompassed w/in the Universal Principles
http://www-128.ibm.com/developerworks/rational/library/5823.html - Extending the RUP
Eclipse Process Framework – www.eclipse.org/epf
OpenUP
open source version of UP. Modification of RUP material. Brief description of RUP (overview). Easy to use and available for free. Anybody can use it. Currently in BETA. "looks like crap – user interface challenges"
A Method Framework is a framework based on a common set of principles.
RUP – gives you everything and you choose what you need
OpenUP – gives you the base and you can add on what you need

Agile UP (AUP) – worth looking at
"The only true measure of progress in a software development project is working software"

Essential UP (Ess UP) – worth looking at
www.ivarjacobson.com
Lightweight
Agile
Freely available
Easy to use
Open source process
Only the "essential" practices
Agile 2.0 process

Enterprise Unified Process – wraps UP into a wider scope process that includes support and enterprise concerns. Change management, project management, operations, support, Portfolio Management, Strategic reuse, People Management, Enterprise Admin, Software Process Improvement, etc.
"Architect also codes" – speaker is a firm believer in this pattern – you need to be involved with the team.
Developers will not read white papers and look at models. Developers will download code. Model your designs in code for developers to use. Make sure examples are high quality. Developers do not respect architects who do not write code.
Speaker believes the industry suffers from too much theory and not enough reality. Discuss how it "should" be done, but do not observe how it is actually done. Example – how many books are there on how to model on paper and white boards? The books are on how to model using tools.

Strategic Reuse – almost always fails. You need to have a reuse plan and encourage it. Charge-back plans for reuse will kill the effort (if one department/team uses components/service from other teams, the developer team charges the consumer internal funny money). Monitor what your teams are doing and when something looks like it could be made reusable, do so. Do not design with the intent of reuse in mind; distracts from the project needs.

Agile UP is the lightest of the options
EAP is the heaviest

AUP and Open UP are emerging – gaining acceptance and support

Suggests we take Open UP seriously – main editing tool is eclipse, but it is NOT eclipse specific

US Department of Defense has belief that all processes can be broken down into granular repeatable steps. US Department of Defense is statistically one of the least successful software development team anywhere.

Scott W. Ambler
www.ambysoft.com/scottAmbler.html
www.agiledata.org/feedback.html
www.agilemodeling.com/feedback.html

Architect Soup - EA, SOA, EDA, SCA, MDA
Mike Rosen – architect for ~15 years
EA – enterprise architecture
SOA – service oriented
EDA – Enterprise Driven
SCA – Service Component
MDA – Model Driven

Enterprise Architecture
About enabling and managing change
Goal is to align IT systems with business goals and strategy

Secondary goals

Reduce IT expenditures
Run IT as a business
Support portfolio management
Support outsourcing
Provide governance framework
Enable SOA

Zachman
Usually implemented with a framework – Zachman is most commonly implemented.
"Technology is not the solution to IT problems, Architecture is." - Zachman
States Zachman is a great way to start, but it is not a THE solution

Federal Enterprise Architecture Framework
Implemented by US Federal Government

Service Oriented Architecture
Most SOA definitions are technology focused, but only address a small part of SOA
Web Services are a good technology for implementing SOA, but not the only. Can use CORBA, Java, .NET
NOT new – CORBA and Tuxedo have been successfully used tools. Many other attempts failed.
Web Services are not SOA
Architecture commonly fails at the delivery of standards and architecture to the developers. Drafting a white paper and other documentation is not as affective as delivering samples and templates.

Event-Driven Architecture
Any app that reacts intelligently to changes in the environmental conditions – failure on a hard drive, sudden change in sales demand
Publish/Subscribe services (Event Management)
Event/Sponsor/Response systems
Applications constructed entirely from "state machine" modules
One where we think about communication between different parts of a business in terms of the occurrence of an event
Workflow needs to be addressed as part of the architecture.

Service Component Architecture
Doesn’t see this getting much traction

Model Based Development
Major initiative from Microsoft and IBM

A side effect of UML Standardization

Create models at business, application, and implementation levels. Write code to support all views of the model.

Model - is a representation of the system
Formal model - is a model that applies to stringent rules.
Model Compiler – can take a formal model and produce transformational output

MDA tools/compilers can generate code for us

Challenge with test generation tools for MDA is they generate TOO Many tests – not all are necessary, but tools can’t discern value of the tests.

Theoretically, models allows you to not be concerned with technology changes. For example, move from .NET 1.1 to 2.0 would be "easy" – build new code base from models for the new platform and you are done.

Introduction to UML

Ok. You might ask why I took this class. First of all, it is immediately after lunch and I am tired; not sure I could take on another heady subject. Secondarily, although I use UML fairly regularly, I’ve never taken a course or read a book on it. I thought maybe I should.

UML is the standard language for visualizing, specifying, construction and documenting the artifacts of a system

http://www.omg.org/ – can download UML from site

The importance of modeling

Smaller projects may not require modeling

easy to build a dog house

Larger projects require modeling

Difficult to build a two-story, five bedroom colonial
Very difficult to build an office complex

Why we model

Communication
Manage complexity
Makes people think
To help understand requirements
To drive implementation
To understand the impact of change
To ensure resources are deployed efficiently

Activity Diagrams

Is a flow chart

Used to show flow of control

Usually used early on in the process
Good for business rules
Flow within a use case
UML 2.0 allows for the interruption of an activity

Use Case Diagrams

Visualization between use cases and actors
Start with Actors

Someone or some thing that must interact with the system under development
Rendered as a stick person (usually)

Use Cases

Why the actor wants to use the system
A pattern of behavior the system exhibits
“A sequence of functions where the outcome makes the actor happy”

Interaction with an ATM is likely one use case

Deposit, withdrawal, transfer, balance inquiry all one use case

Use case diagrams can have includes and extends

Includes – one use case or piece of a use case includes another

Make reservation includes search for flight

Extends – one use case or piece can be extended by another

Select seating location extends Make reservation

Do not overuse these

Can lead to functional decomposition
If not sure, create separate use cases

Nothing new in UML 2.0

Interaction Diagrams

Show dynamics of the system

Show communication between things

Includes Sequence Diagrams

Sequence Diagrams

Should show distribution of behavior between objects
Should not have a lot of sequences pointing to one object
Can get large very fast
Does not represent conditionals very well (if, then, else)

Addition of frames allows me to make composite diagrams – one sequence diagram can include another by reference

Looping and Breaking are now represented well (better)
Negatives, assertions, and critical regions are available

Communication Diagrams – changed in 2.0 – was called something else

Timing Diagrams – added in 2.0

Interaction overview diagram – flow between interactions. Could be represented by activity and sequence diagrams. Not sure they are needed….

Class diagrams – show static structure

Collection of objects
Want to have standards for naming of classes
Classes should have all operations

Look at sequence diagrams to determine required operations
Known operations not in a sequence diagram = missing sequence diagram

Classes have attributes
Classes have relationships

Not required, but a system with classes and no relationships is not possible
Association, aggregation, composition, dependency

Model all as association first

Relationships are discovered by examining interaction diagrams
Multiplicity

How many objects participate in a relationship

One to one
One to many
One to zero or more
Etc.

Navigation

Indicates directionality of communication

Arrow states uni-directional

Want to have as many as you can uni-directional

Inheritance

Relationship between class and subclass

Realization

No change in UML 2.0

Composite Structure Diagrams – new in 2.0

State Diagrams

Shows life history of a given class
For objects with significant dynamic behavior

Component Diagrams

Components can be logical or physical
UML 2.0 – components can have ports and notation has changed

Artifacts are new to UML 2.0 – represents a physical entity

UML Extension

You can extend the UML for things like Databases, business processes, web pages, etc.
Stereotype

Martin Fowlers UML books are good

Scott Amblers process books are good

Has heard UML 2.0 in a nutshell is good – not fond of UML 2.0 for dummies

AD World - Day 1

2006-07-17T08:08:00.000-04:00

Registration
Ok, registration actually took place yesterday. We arrived early and decided to go ahead and pre-register to make things easier on ourselves.

Talk about easy... they did not ask for any form of ID. We walked up to the counter, looked down at the name tags all layed out and easy to read, said a name, and were handed a lanyard, badge, bag, shirt, and CDs. No questions asked. Nothing to sign.

Not a great first impression.

Designing Service Oriented Applications
Service Oriented Architecture

SOA History
CORBA, Tuxedo – prior platforms/tools that SOA has been implemented in. Many other attempts with other tools (and these) failed.

Case Study (loose)
Created a group called the Business Objects Service Group and they were assigned the task of creating reusable services. Service talented developers were put into BOSG. Other (front-end) developers placed in business units to create apps that consume services.

Separation of Function from Interface put the company at a competitive advantage – able to react to change in the market place much faster (took time to build library of services) Doesn’t happen overnight.

System developed in CORBA – now in EJB. Handles 2 Billion transactions/month

Proper architecture requires more than one architectural view
Make it easy to enable the developers to build applications that follow the architectural standards

SOA is concerned with the independent construction of services that can be combined together

Service – encapsulates a unit of work, made available through a service contract
Dynamic Binding, Loose Coupling, Modular and self contained, composable

BPEL – Business Process Execution Language – used inside a business service; usually provided by the technology platform

Business Process Level (BPM) – manage process; checkpoint, auditing, security, etc. also determine sequence/series of services changed together to achieve a need

Salesforce.com, eBay, Amazon, WebSuite – provide third party Web services

Majority of corporate expenditure for application development is on Maintenance. Next highest number is on Integration. Last is on New development.

Coupling
Loose Coupling – Want to make it possible for the creator of the service to make adjustments without requiring the consumer to alter their use of the service. Decouple the life cycle of the consumer from the life cycle of the provider.

Synchronous vs. Asynchronous – not really significant in SOA. These terms are more specific to the requirements of the system, but not specific to SOA and decoupling. Most SOA implementations will use both.

Interface and Implementation – allows a service to change without requiring changes of the service consumers. Important in an SOA architecture. SOA ensures the interface contract is the ONLY means of interaction.

Publication, Discovery and Binding
Registry exists as a broker
Service is published as a service definition to the Registry
Service Provider registers as a provider of an implementation
Consumer requests a service from the registry
Registry makes a decision and provides “handle” to the service.
Consumer then connects to service

SOA integration
Integrate Once, use many, consistent access, lower total cost
Adaptable to change
Incremental approach – start small, add new integration services over time as part of specific projects
Flexibility increases with each new service

SOA and Web Services by Eric Newcommer – “Not a bad book” – decent book RE: Web services, but weak regarding SOA overall.

Best Practices in SOA Development

MDA Overview

Model Driven Architecture

Business Model -> Application Model -> Implementation Model -> Code

Model – a representation of the system. A model describes part of the function, structure and/or behavior of a system.

Formal Model – a representation of the system conforming to rigorous rules All Model Based Development models are formal

Model should be platform independent – independent of the language/technology used to build it. Also independent of any hardware.

PSM – platform specific

PIM – Platform independent

Tools exist that can generate code based on well defined models. Use of these tools is convenient and saves cost, but regardless, the process of building models accurately (theoretically) makes app dev/maintenance easier and more flexible.

Applying MDA to SOA
More SOA app initiatives are bottom up rather than top down. Top Down is based on a Business Process Model. Bottom Up is trying to leverage an existing service or service enable legacy systems. Best approach is “middle out”

Process Overview
· Understand Use Cases – not too detailed
· Specify Scenarios – show flow of use cases with actors
· Design/Align Information Model
· Understand Overall Context
· Look for appropriate patterns
· Identify Service and Interface Style
· Define Documents
· Define Service Implementation
· Look for appropriate patterns
· Information Transformation follows data flow

SOA should implement the most important things first; shared information and shared functions.

Threat Modeling: Creating Secure Applications
Agenda is based on two books:
Writing Secure Code and Threat Modeling

Threat Modeling (book) – good place to start, but getting long in the tooth. Great if you are just getting started.

void main() {
char buf[512];
gets(buf);
}

The above is vulnerable to a buffer overflow.

Java Principle – As you work on an application, it gets better and better.
This is a fallacy. The likelihood of bugs is no more or less with each release.

“Security is all about data. Attacks are via data. What they are looking for is data.” therefore
“The ultimate safe program is one that takes in no data and produces no results.”

What are threat personas?
A systems’ anti-users
- based on real-world data. There are five behaviors and eight actual personas

“One person’s feature is another person’s exploit”

Author
Vandal – Curiosity; usually script kiddie
Trespasser – after personal fame – want to make a mark and get credit for it - usually hobbyist/hacker
Personal Gain – Thief – actually after the data to take advantage of it – usually hobbyist, but many expert
National Interest – spy – experts and specialists

Some Important Definitions
Threat Agents – someone who could do harm to a system (adversary)
Threat – An adversary’s goal
Threat Tree – A graphical representation of security-relevant pre-conditions in a system
Vulnerability – a flaw in the system that could be exploited
Asset – Something of value to the valid users and adversaries alike
Attack – When a motivated and sufficiently skilled threat agent takes advantage of a vulnerability

Threat Model – describes a system’s threat profile

Classic Threat Modeling
Collect Background Information -> Model the System -> Determine Threats

Collect Background Information
Identify Scenarios
how the system is intended to be used (or not intended) in deployment
Help us understand what was considered in security model in the first place
Identify External Dependencies
Dependencies / requirements outside of our system/model
Implementation Assumptions
Assumptions should be validated on completion of implementation
Identify external security notes
Counterpart to external dependencies
Identify internal security notes
Makes model more clear
Explain tradeoff made in design or implementation of the system that affect security

Model the System
Modeling is critical to determining threats; helps us understand adversary’s view of the system; helps team understand internal workings
Identify Entry Points
Define the boundaries of the system being modeled
List all places where the system consumes or provides data including actions on behalf of third parties
Often obvious, but may not be. Reading files from the system is an entry point.
Identify Assets
Those things (concrete and abstract) that could be targets of an attack by an adversary
Assets should be nouns
Assets can be widely varies; data in a database, network coherency on a peer to peer application
Identify Trust Levels
Have preconditions (authentication)
Trust levels characterize either entry points or assets
Trust level specific to the entry point of asset
Create flow diagrams / process models
Describe processing on the threat path
Threat Path is the processing that occurs based on the input to the enumerated entry points
Follow the processing from entry point – determine what it does (or can do to data)
Create Data Flow Diagrams
Context Diagram
Keep modeling until there are no more multi-processes
Building a DFD
Identify the actors and where data moves in and out
Break-down HOW data moves in and out
Keep these at the high level

Determine Threats
Enumerate threats – creates a threat profile
Threats with valid attack paths are vulnerabilities
Threats are verbs

STRIDE – types of threats to a system (all known threats to date fall into these types)
Spoofing – pose as another user
Tampering – modification of data
Repudiation – deniability of malicious acts
Information (disclosure)
Denial of Service
Elevation of Privilege

DREAD – means of characterizing the risk associated with a vulnerability
Damage Potential – extent of possible damage
Reproducibility – how easy is it repeat the attack with success
Exploitability – Effort required to execute the attack in the first place
Affected Users –ratio of installed instances affected is exploit is widely available
Discoverability – how likely the unpatched exploit is to be found by others

CIA – Confidentiality / Integrity / Availability

Threat models should include review from outside parties.

Conclusion

So far, so good. The SOA class was too slow to start and too quick at the end. Bored me and then lost me. Security course was very good. Lots of useful tools and examples. Nice exercise. I'm definitely growing more and more interested in this aspect of our field....

Writing Secure Code - Authentication

2006-07-02T20:48:00.000-04:00

There are numerous authentication mechanisms to choose from. If not correctly selected and implemented, the authentication mechanism can expose vulnerabilities that attackers can use to gain access to your system.

Vulnerabilities
Network Eavesdropping
If authentication credentials are passed in plaintext, an attacker armed with basic network monitoring software on a host on the same network can capture traffic and obtain user names and passwords.
Note: This type of attack implies the network has already been compromised.

Brute Force Attacks
Brute force attacks rely on computational power to crack hashed passwords or other secrets secured with hashing and encryption. Brute force attacks typically involve trying to work through all possible key combinations to decrypt a string and are, therefore, difficult to accomplish.

Dictionary Attacks
This attack is used to obtain passwords held as hashes. On such systems, users are authenticated by re-computing the hash based on the user-supplied password value and comparing it against the system-stored hash value. If an attacker manages to obtain the list of hashed passwords, a brute force attack can be used to crack the password hashes.
With the dictionary attack, an attacker uses a program to iterate through all of the words in a dictionary and computes the hash for each word. The resultant hash is compared with values in the data store. Weak passwords such as “Yankees” or “Mustang” will be cracked quickly. Stronger passwords such as “RuN4PhuN”, are less likely to be cracked.

Cookie Replay Attacks
With this type of attack, the attacker captures the user’s authentication cookie using monitoring software and replays it to the application to gain access under a false identity. See Session Hijacking and Replay and Man in the Middle Attacks for more information.

Credential Theft
If your application implements its own user store containing user account names and passwords, compare its security to the credential stores provided by the platform, for example, a Microsoft Active Directory® service user store. Current implementation uses Active Directory® as the user store, but Gemstone should not rely solely on the fortitude of the chosen user store.
Browser history and cache also store user login information for future use. If the terminal is accessed by someone other than the user who logged on, and the same page is hit, the saved login will be available.

Recommendations
· Secure the channel
· Encrypt it
· Strong authentication requirements
· Avoid holding long sessions
· Re-Authenticate

Writing Secure Code - Input Validation

2006-07-02T19:45:00.000-04:00

There are a number of attacks an intruder can use that take advantage of the common assumption that data supplied by users while filling out a Web form is safe. Improper checks against data supplied by users can make the site vulnerable to a number of different attacks.

Vulnerabilities

Buffer Overflows

Buffer overflow attacks have been around for decades. A buffer overflow attack can either result in a denial of service or it can cause code injected by the attacker to be run on the server. .NET code is not as susceptible to buffer overflows because the code base is better managed and array bounds are checked before arrays are accessed. Even .NET sites can be susceptible where unmanaged APIs or COM objects are involved.

Cross Site Scripting

In Cross Site Scripting (XSS), an attacker takes advantage of poor handling of data either at entry or display. This technique can be used to gather confidential user information or to impersonate users and achieve access to the Web application with the same rights as the impersonated user. Cross Site Scripting is a technique commonly associated with phishing.

SQL Injection

In SQL Injection, and attacker takes advantage of poorly handled data and weakly constructed queries to a SQL Database. Most commonly, this is on screens where developers are relying on input from the end user to filter or sort data and then the data provided by the end users is not properly validated.

Recommendations

Validate all input all the time

Assume all input is malicious, regardless of source and handle it as such. You can’t be certain that a service, file share, or database you work with has not been compromised. You absolutely can’t be certain that a user is who they claim they are or has good intentions.
Do not assume that data validation only need to take place at a single layer of the application. Verify data at all levels of the application. If any one layer is circumvented or compromised, the remaining layers must perform their due diligence to assure the integrity of the system.

Use common validation routines

Make input validation a core element of your application development strategy. Create shared validation routines for all common routines such as email, zip code, phone numbers, etc. This ensures validation is consistent and makes maintenance much easier.
Be careful about page or module specific validation. Make sure this approach is truly necessary and then attempt to leverage as much of the common routines as possible.

Constrain and sanitize

Constrain
To constrain data is to allow only expected characters or patterns to be submitted. This is commonly accomplished through the application or regular expressions. However it is applied, the idea is to check the data for type, length, format, and range, considering all data that fails to meet the criteria as bad. We would check string patterns and reject any that did not match our specific rules. This not only eliminates errant characters, but further assures the accuracy of the data stored.
In the case of an age field, for example, the length would be at least one and no more than three and only digits would be acceptable. Any string not matching this pattern would be rejected.

/^[0-9]{1,3}$/

An email address would be more complicated. The following pattern is good for most email addresses. It must start with a character, followed by any number of word characters, dots, or hyphens, followed by either a character or digit, followed by “@”, followed by either an IP Address or a character, followed by any series of word characters, dots, or hyphens, followed by either a character or a digit, followed by a dot, and two to four characters.

^[a-zA-Z][\w\.-]*[a-zA-Z0-9]@([0-9]{2,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})([a-zA-Z][\w\.-]*[a-zA-Z0-9]\.[a-zA-Z]{2,4})$

Sanitize
To sanitize data is to transform the data into a safe format. This is different than constraint. In constraint, we do not allow data that does not match our patterns. In sanitization, we alter the data to ensure it is not harmful. This may include stripping nulls or other extended characters from strings or escaping out values so they are treated as literals.
In the simple example shown below, we remove the characters, whereas in actual code, we may choose to replace the characters with displayable representations, such as replacing “<” with “<”.

function RemoveBad(InStr){
InStr = InStr.replace(/\/g,"");
InStr = InStr.replace(/\"/g,"");
InStr = InStr.replace(/\'/g,"");
InStr = InStr.replace(/\%/g,"");
InStr = InStr.replace(/\;/g,"");
InStr = InStr.replace(/$/g,"");
InStr = InStr.replace(/$/g,"");
InStr = InStr.replace(/\&/g,"");
InStr = InStr.replace(/\+/g,"");

return InStr;
}

The HTMLEncode method will escape out HTML Characters and the URLEncode method will ensure a URL is a valid URI request. These should be a required part of your standard input/output data handling.

Set the Character Set

If the character set of a page is not explicitly defined, the server is unable to determine which characters are special. This ambiguity can be exploited by hackers because filters for special characters are that much more difficult to create.
Character coding for HTML and HTTP was intended to default to ISO-8859-1, but many browsers did not support this encoding by default. Version 4 of the HTML standard now allows for any character encoding to be used, unless explicitly indicated in the page header.
Recommendation is to set all pages to the same character set, consistent with the server. The following shows a simple example of how to set the character set to ISO-8859-1 in an HTML page. This can be done through a more universal means such as a standard include file for all page headers or use of page templates.

<html>
<head>
<META http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>HTML SAMPLE</title>
</head>
<body>
<p>This is a sample HTML page</p>
</body>
</html>

Wine and Friends

2006-07-02T18:25:00.000-04:00

For those of you who know me, you know I am not much of a sophisticate. In fact, I am to sophisticate as Keanu Reeves is to extraordinary thespian. If you don't know any better, you might mistake either of us as our respective pairing.

This past weekend was a wonderful experience for my wife and me. Some friends invited us out to dinner and a wine tasting. Now, as fairly recent graduates from the school of "White Zinfandel is the only wine I know", we were in over our heads. And I was intimidated. We were to meet a third couple for the dinner. Both of the other couples are quite well versed in wines.

We explained very early in the evening that we knew little, if anything, about wine. Once confessed, I felt better. At least I didn't have to pretend. But much to our delight, they were more than happy to help us learn. Through the entire five course meal, they explained the origins of the various wines, the flavors, and the proper pairings with particular foods. They showed us how to take in the boquet, swirl, sample the boquet again, and finally taste.

Of course one evening does not make us experts. This is certainly an area of knowledge where the more you know, the more you realize how little you know. It takes years to develop a proper pallet and decades to become an expert.

But it took only one evening to make good friends.

Zoho Writer

2006-07-01T14:32:00.000-04:00

Most of you have heard of Google Spreadsheets. If you haven't, you should certainly check them out. For the Excel officianado (Mr. oz108us, you know who you are), the Google sheets will be nothing but an exercise in frustration and limited functionality. But for us commoners who don't do much beyond keep lists and 0ccassionally SUM() or AVG() a column, they rock! And they are far more affordable than Excel.

Now I've found Zoho Writer. Currently a free service, Zoho Writer allows you to draft documents on-line with nothing but a browser. I took a document I had written in Word, copied and pasted it into Zoho and the content transferred perfectly, formatting, style definitions, and ALL. Very nice.

Something is wrong with the system!

2006-06-20T21:57:00.000-04:00

Once again inspired by something I found on the Daily WTF.

A few years ago, I worked with a team of developers on their in-house order entry system. The system had been written over the course of several years with no real architectural plan. Just build it as they ask for it. Of course, the system was wrought with issues. But, it did the job well enough and the company was convinced they were so unique in the way they took and filled orders, that this piece of custom software was the ONLY solution they would ever be able to use.

When I got involved, the team was in a maintenance phase between projects. They were slowly ticking away at the literally thousands of bug reports from customers; addressing them one at a time in no particular order.

We decided to take a slightly different approach and look at the bug lists from a 10,000 foot view. We lumped bugs together into various categories and discovered that record locking was the root cause of MANY of the issues. So we set about creating a standard for record locking and transaction management. We then tackled this one problem everywhere it existed, starting with the most critical code first.

After some time, we had made a significant dent in the code and our tests were looking good. In fact, CPU utilization in the test and QA environments appeared lower. But the real test would be Production.

We rolled the first wave of changes out to Production, and as was tradition, the developers crossed their fingers, waited for the phone calls, and readied themselves to do battle, real-time, with code in Production.

Hours passed without a single complaint. Finally, near the end of the day, helpdesk received a call from order entry.

"Something is wrong with the system." the caller reported, "I think we're losing orders!"

"What makes you think orders are lost?" asked the helpdesk staffer.

"Well", explained the caller, "we usually get a message every hour or so telling us the order table can't be updated because somebody else is using it. This happens anytime several of us are taking calls. We just wait a minute and then hit 'OK' again and it works"

"Yes...?"

"Well, I checked with the other folks here and nobody got the message today. Not one of us. But we were all taking calls and placing orders. The system must be losing some of the orders if it doesn't know we're all putting them in."

Yes, folks, the system was "broken" because it wasn't generating enough errors.

Staffing for Web Development

2006-06-20T21:42:00.000-04:00

I just read a decent article on how to plan the size of your team for Web development. The article discusses the atributes of a site that can be used as indicators of staffing needs; size, complexity, and activity. Of course, a large, transactional site with millions of visitors per day would need a large staff to keep it running.

It occurred to me as I was reading the article, that I have never worked anywhere that provided adequate staffing for the continued development and maintenance of their Web sites.

One employer had over 100 people on staff dedicated to a single site. That sounds like enough, but the truth was we struggled to keep up with the demand for new development much less had time to do maintenance and refactoring correctly.

It seems to me that this very phenomenon is a root cause for the number of sites vulnerable to hacks months if not years after they've been made public and standards have been written to block against them. One can still get into some pretty serious sites with basic Cross Site Scripting.

But, I digress... I think. My original point was simply that companies regularly understaff their Web efforts. Shane Diffily's article may help you build your case for the extra staffing you know you so desparately need.

Interesting Stances on Enterprise Architects

2006-06-11T20:39:00.000-04:00

Setting Values versus Enforcing Standards
http://scottmark.blogspot.com/2006/02/setting-values-versus-enforcing.html

Enterprise Architects Versus The World
http://enterprisearchitect.typepad.com/ea/2006/04/enterprise_arch.html

Differences Between 'What' & 'How' in Enterprise Architecture
http://enterprisearchitect.typepad.com/ea/2006/03/differences_bet.html

Resources

2006-06-06T19:53:00.000-04:00

A list of resources provided by my friend Eric.

Blogs:

Andrew Hunt (agile manifesto signer & pragmatic programmer) http:// www.toolshed.com/blog/
Dave Thomas (agile manifesto signer & pragmatic programmer) http://blogs.pragprog.com/cgi-bin/pragdave.cgi
Char Fowler http://www.chadfowler.com/
Scott Guthrie (manager for VS2005, IIS and ASP.NET) http://weblogs.asp.net/scottgu/
Nikhil Kothari (one of Scott G's minions working on Atlas) http://www.nikhilk.net/
Rockford Lhotka http://www.lhotka.net/WeBlog/
Martin Fowler (agile manifest signer & software architect) http://martinfowler.com/bliki/
Eric Sink http://software.ericsink.com/
David Hayden http://www.davidhayden.com/davidhayden/
Joel Spolski http://www.joelonsoftware.com
Darrell Norton http://codebetter.com/blogs/darrell.norton/
Scott Hansel http://www.hanselminutes.com/
TestDriven.NET Blog http://weblogs.asp.net/NUnitAddin/
Looks Good Works Well http://looksgoodworkswell.blogspot.com/
Don Smith (Dev Consultant at Microsoft specializing in Web Services) http://blogs.msdn.com/donsmith/
Technology Architecture & Projects http://enterprisearchitect.typepad.com/ea/
Nick Malik (Enterprise Architect for Microsoft) http://blogs.msdn.com/nickmalik/
Dominic Baier (.NET security oriented stuff) http://www.leastprivilege.com/

General Info:

TheServerSide.NET http://www.theserverside.net
O'Reilly Windows DevCenter http://www.windowsdevcenter.com/
.NET Architecture Center: SOA http://msdn.microsoft.com/architecture/soa/default.aspx
PatternShare Community http://patternshare.org/
SOA Hub http://www.soahub.com/

Tools:

Vil (provides metrics, visualization, querying, and analysis of .NET assemblies, classes, and methods) http://www.1bot.com/

Simian (similarity analyzer reporting) http://www.redhillconsulting.com.au/products/simian/
NCover (unit test code coverage reporting) http://ncover.org/site/
NDepend (code dependency reporting/visualization) http://www.ndepend.com/
TestDriven.NET (formerly NUnitAddin) http://www.testdriven.net/
Refactor Pro (killer refactoring tool) http://www.devexpress.com/Products/NET/Refactor/
NAnt (build tool like BusyBee but supported by the author) http://nant.sourceforge.net/
CruiseControl.NET (continuous integration engine) http://ccnet.thoughtworks.com/
GhostDoc (inline doc helper) http://www.roland-weigelt.de/ghostdoc/
Rhino Mocks (fantastic mock object engine) http://www.ayende.com/projects/rhino-mocks.aspx
Selenium (automated web app testing tool) http://www.openqa.org/selenium/
IeUnit http://ieunit.sourceforge.net/
PesterCat (automated web app testing) http://www.pestercat.com/
SW Explorer Automation (automate IE to run web app testing) http://home.comcast.net/~furmana/SWIEAutomation.htm

Libraries:

AJAX & Effects (specifically Javascript) Related:

Prototype (javascript framework) http://prototype.conio.net/
Dojo (like Prototype, but different approach) http://dojotoolkit.org/
Javascript Windows http://blogus.xilinus.com/pages/javawin
OpenRico (behavior/effect library works with Prototype) http://openrico.org/
Script.aculo.us (behavior/effect library works with Prototype) http://script.aculo.us/
AJAX Libraries tracker: http://mail.interaktonline.com/ajax/libs/index.php

AJAX (server side) Related:

AJAX.NET Professional http://www.schwarz-interactive.de/
Microsoft Atlas http://weblogs.asp.net/atlas
Microsoft Atlas related http://www.nikhilk.net/

Correcting Browser Image Flicker

2006-05-31T20:49:00.000-04:00

I was reading Slash Dot today and noticed that any time I ran the mouse over the logo, it flashed; how annoying.

I asked a colleague about it and he suggested I change the settings on my browser, because it had to do with image cache. While I could adjust my browser, it seemed there must be another way.

It turns out you can adjust the settings on your server to set default caching requirements for graphics and this will fix the problem, but it didn't seem right that I would have to ask my hosting provider to change server settings either....

So a little more searching and I found the solution (assuming you're using ASP.NET). You can create an HTTP handler for images. I'm not going to go into the details here, since this article is all you need.

How to make that change

2006-05-31T20:18:00.000-04:00

If you want to change something in your life, it's common to try to stop the behaviors you don't like. While this certainly seems logical, it seldom works. The reason is simple - you create a vacuum where the old behaviors used to be. Since nature hates a vacuum it will fill the new void with anything it can find - typically the very behaviors you're trying to stop (since they're so familiar). Instead of stopping certain behaviors, try focusing on what you want to create and begin using the new behaviors. Over time, the old behaviors will be replaced by the new.

One place this idea can be important is in changing your management style. If you are abrupt or abrasive with employees and WANT to change the behavior, focus on being patient and pleasant rather than focusing on stopping the undersireable behavior.

What about weight loss? How many people focus on what they have to stop doing in order to lose that extra weight; quit snacking so much, don't eat such large meals, no snacks before bed, etc. Focus on what you should do instead; eat smaller meals more often during the day, eat fresh fruits, eat fresh vegetables, etc.

Try it yourself:
1. Identify something in your life you feel you've got to stop doing.
2. Think about what you need to start doing in that area.
3. Be specific and write down the specific new behaviors.
4. Don't punish yourself for the old behaviors. Stay committed to the new ones and the old ones will eventually disappear.

Real Fight Clubs

2006-05-30T21:52:00.000-04:00

URL: Techies beat each other bloody at fight club

Wow. This is disturbing. I've met maybe five people in my 15+ years in IT that I suspect would willingly attend one of these events. And I am sorry, but I do not believe this is a latent behavior learned from video games nor do I think this is anything close to normal. This isn't about being a man, this is something else entirely. Sparring with a trained karate partner is not the same as bare knuckle brawls for "fun."

I loved the movie Fight Club. It was edgy and a great fantasy. But it was fantasy and nothing more. If I recall, the guy that started the whole thing ended up to be a nut job who killed himself. Not your typical hero.

This, ultimately, is few dozen men in serious need of therapy.

Happy Birthday

2006-05-29T21:16:00.000-04:00

Happy Birthday to my wonderful wife!

Today is my wife's birthday. We did not do much celebrating as we spent the day driving home from Chicago where we visited with my parents and grand parents. I drive for a few hours, my wife drove for a couple, and our daughter drove for a couple. I think the last part, where my daughter drove (she's 16), was most taxing on my wife.

Anyway, I wanted to tell my wife in front of witnesses and the cyber world just how much she means to me. We've been married for 17 years and I look forward to seeing her every day. She is my strength, my lighthouse in the storm. She always provides me the right path, no matter how lost I feel and she is with me even in my lonliest moments.

Jennifer, you are most precious to me. And I love you.

Always.

Michael

Interesting Take On Gender

2006-05-27T19:50:00.000-04:00

Thank you to my friend Leonid for sharing this one.

One of Leonid's customers has a home-grown CRM system. They asked him to perform some analysis on the database for code inefficiencies, vulnerabilities, etc. While reviewing the system, he came across a table named "SEX_TBL" and, of course, was just too curious.

A simple query showed four rows in the table:
Male
Female
Unknown
Group

Male and Female seem self-explanatory. Even unknown in these modern times makes sense, but Group Sex always meant something entirely different to me....

ReSharper 2.0 Released!

2006-05-26T05:58:00.000-04:00

Ok, so I'm a little slow on this one....

JetBrains has released the 2.0 version of their Visual Studio add-in. For those of you who are unfamiliar with ReSharper, it provides error highlighting, unit test integration, and support for code refactoring all from the Visual Studio IDE. Certainly worth a look.

You can download a 30-day trial from the JetBrains site.

Injection Rejection

2006-05-25T19:54:00.000-04:00

I've been studying Cross-Site Scripting and SQL Injection lately for a security project I am working on. I found this article on one of my favorite "free time" sites and thought I would share.

Thanks to Alex Papadimoulis for his site The Daily WTF.

URL: Injection Rejection

ASP Impersonation

2006-05-25T19:48:00.000-04:00

Had an interesting challenge over the weekend. I received a call from an individual who is running a web site written in ASP. The site started out as a simple HTML site, but slowly grew into something more and is now a form of revenue for him. I won't get into any more detail than this about the specific site - for his own protection. ;)

Again, skipping some details, the site authenticates users against a database and then sets a cookie to indicate the user is authenticated. The challenge was that a specific page in the site needed elevated rights in order to update some text files within the site - think of it as one web page writing another.

Anyway, this posed a serious problem for him. The users are not actually authenticated against the server as NT users, so everything they do is under the IUSR_MACHINE user account. In order to be able to update text files, he elevated the rights of the IUSR_MACHINE user. he then was concerned he may have granted all of his pages too much access to his machine.

So how to get around this?

In ASP.NET, it is relatively easy, but with ASP, we have a challenge.

After some digging, I found a simple way to write a DLL that does the lifting for you and allows you to temporarily act as another user.

We implemented the spiffy new DLL, lowered the rights of IUSR_MACHINE and created a separate user with the rights desired. The page now impersonates the desired user and then reverts back to IUSR_MACHINE when done.

This post will help you get started:
http://support.microsoft.com/default.aspx?scid=kb;EN-US;248187

Excellent Series on Management

2006-05-23T17:59:00.000-04:00

Prior to my new assignment of Senior Application Architect, I was a Software Development Manager. I loved that job and I took it quite seriously. Having been recently hired into a tumultuous environment where upper and middle management was changing every 2 months (I was unaware of this phenomena until AFTER I was hired), I found myself with a relatively new and dysfunctional team. I struggled for a bit, trying to get them to come together as a unit, but I discovered a series of books written by Patrick Lencioni that helped a great deal.

These books may seem too simple at first. As a manager, you are faced with complex problems. A simple parable with simple lessons could not possibly hold the answers to your unique and difficult situation. The lessons learned in these books are not new and they are quite simple. Most important, they are effective.

Anyway, I recommend them for a read.

The Five Dysfunctions of a Team
Overcoming the Five Dysfunctions of a Team
Death By Meeting
The Five Temptations of a CEO

He has others, but I have yet to read them.

I will be concentrating more on technology and less on people as my new role demands. I do hope someday (soon) I will be able to put my focus on people once again. Perhaps I will find an employer who truly values people and recognizes that it is through leadership, not management, that the best results are achieved.

How Would You Feel?

2006-05-22T20:15:00.000-04:00

About a week ago, I was informed that my bright shiny and rewarding job of Software Development Manager was being eliminated in an organizational flattening. This was a "no fault" move - I had done nothing wrong, the organization was simply too management heavy. As a result, I was being provided the opportunity to select from any one of three different jobs to fulfill; Lead Software Developer, Lead Project Manager, or Senior Application Architect. I was to receive the same compensation for the new role.

Many of you may say, "That is unfortunate, but at least you are still employed and did not get a cut in pay." While I recognize both of these factors, I struggle to find genuine tiding in the situation and certainly no comfort.

Less than one month prior to this flattening of the organization, a new manager was moved into our group. There was no requisition open. We had no need. He brought with him only two employees and no defined set of responsibility. He had, however, been with the company for ten years compared to my one. He had been a manager in another division of IT, but had grown weary of the struggles of the role and asked that he be placed somewhere else. This gentleman is now my boss and has inherited my 17 person organization as his own.

It is the orchestration of these events that trouble me. There was no open position or need in our group for another manager. We were already manager heavy and we knew it. He was unhappy in his role and wanted something else. I suspect he makes a great deal more than I and could not have been easily moved into a lower pay-grade without a salary adjustment or extremely limited opportunity for raises and incentives.

Should they reward his loyalty? Certainly. Should I or anyone else have to lose their position because he elected to step down from his own? I think not. And to add insult to injury, he is now my boss.

I will work to fulfill my new role to the best of my abilities. But for this company to ask or expect my loyalty from this point forward is perverted.