Related link: ftp://ftp.freebsd.org/pub/FreeBSD/SOC2005/bsdinstaller/

As part of FreeBSD’s participation in Google’s summer of code 2005, a new installer is in the works. Here’s a sneak peek at the first beta.

In my RSS feeder this morning there was an article from the DragonFly BSD Digest stating that the first BETA of FreeBSD’s new installer was available for download.

I had a chance to burn the ISO and try it out. For those of you hoping for a pretty GUI installer, don’t look here–you probably want something more along the lines of PC-BSD. For the immediate time being, FreeBSD will continue to use an ncurses installer so those users who don’t want to install X won’t have to just to install the operating system.

While it’s just the first beta, there are already improvements over sysinstall. What follows are the screens I saw in this beta version.

—

F10=Refresh Display

Welcome to FreeBSD

Welcome to the FreeBSD Live CD.

FreeBSD is an efficient and elegant BSD Unix-derived operating system. for
more information, see http://www.freebsd.org

From this CD, you can boot into FreeBSD “live” (without installing it) to
evaluate it, to install it manually, or to troubleshoot problems with an
existing installation, using either a command prompt or menu-driven
utilities.

Also, you can use this automated application to assist you in installing
FreeBSD on this computer and configuring it once it is installed.

< Install FreeBSD > < Configure an Installed System >
< Live CD Utilities > < Exit to Live CD > < Reboot this Computer >
< Setup NetBoot Install Services >

—

Begin Installation

This experimental application will install FreeBSD on one of the hard disk
drives attached to this computer. It has been designed to make it easy to
install FreeBSD in the typical case. If you have special requirements that
are not addressed by this installer, or if you have problems using it, you
are welcome to install FreeBSD manually. To do so select Exit to Live CD,
login as root, and follow the instructions given in the file /README .

NOTE! As with any installation process, YOU ARE STRONGLY ENCOURAGED TO
BACK UP ANY IMPORTANT DATA ON THIS COMPUTER BEFORE PROCEEDING!

< Install FreeBSD > < Return to Welcome Menu > < Exit to Live CD >
Press F1 for Help

—

Select Disk

Select a disk on which to install
FreeBSD

< ad0 >
< ad2 >
< Return to Begin Installation >

—

How Much Disk?

Select how much of this disk you want to use for FreeBSD.

ad0

< Use Entire Disk > < Use Part of Disk >
< Return to Select Disk >

—

Are you absolutely sure?

WARNING! ALL data in ALL partitions on
the disk

ad0

will be IRREVOCABLY ERASED!

Are you ABSOLUTELY SURE you wish to
take this action? This is your LAST
CHANCE to cancel!

< OK > < Cancel >

—

Information

The disk

ad0

was formatted.

< OK >

—

Create Subpartitions

Set up the subpartitions (also known as just ‘partitions’ in BSD
tradition) you want to have on this primary partition.

For Capacity, use ‘M’ to indicate megabytes, ‘G’ to indicate gigabytes, or
a single ‘*’ to indicate use the remaining space on the primary
partition’.

Mountpoint Capacity
[/ ] [256M ] < Ins > < Del >
[swap ] [2048M ] < Ins > < Del >
[/var ] [256M ] < Ins > < Del >
[/tmp ] [256M ] < Ins > < Del >
[/usr ] [8192M ] < Ins > < Del >
[/home ] [* ] < Ins > < Del >
< Add >

< Accept and Create > < Return to Select Disk > < Switch to Expert Mode >

—

Select Distributions to install

Select the distributions to Install.
The base is required, the rest are optional

Install? Distribution name
[X] [Base ]
[X] [Documentation ]
[X] [Games ]
[X] [Manual pages ]
[X] [Cat Pages ]
[X] [Profilling Libra] (sic)
[X] [Dictonaries ] (sic)
[X] [Info Pages ]
[X] [Local ]

< Accept and Install Distributions >
< Return to Create Subpartitions >

—

Install OS

Everything is now ready to install the actual files which
comprise the FreeBSD operating system on the selected partition
of the selected disk.

Note that this process will take quite a while to finish. You
may wish to take a break now and come back to the computer in a
short while.

< Begin Installing Files > < Return to Select Distributions >

—

Install Bootblock(s)

You may now wish to install bootblocks on one or more
disks. If you already have a boot manager installed,
you can skip this step (but you may have to configure)
your boot manager separately.) If you installed
FreeBSD on a disk other than your first disk, you will
need to put the bootblock on at least your first disk
and the FreeBSD disk.

Disk Drive Install Bootblock? Packet Mode?
[ad0 ] [X] [X]
[ad2 ] [X] [X]

< Accept and Install Bootblocks > < Skip this Step >

From F1:

‘Packet Mode’ refers to using newer BIOS calls to boot from a partition
of the disk. It is generally not required unless:

- your BIOS does not support legacy mode; or
- your FreeBSD primary partition resides on a cylinder of the disk
beyond cylinder 1024; or
- you just can’t get it to boot without it.

(I tried skip)

—

FreeBSD is Installed!

Congratulations!

FreeBSD has successfully been installed on this computer. You may
now proceed to configure the installation. Alternately, you may
wish to reboot the computer and boot into the installed system to
confirm that it works.

< Configure this System > < Reboot >
< Return to Welcome Menu >

—

Configure an Installed System

The options on this menu allow you to
configure a FreeBSD system after it has
already been installed.

< Select timezone >
< Set date and time >
< Set root password >
< Add a user >
< Configure network interfaces >
< Configure hostname and domain >
< Set keyboard map >
< Set console font >
< Set screen map >
< Install extra software packages >
< Remove software packages >
< Return to Welcome Menu >

—

When finished, I returned to the welcome menu where I could choose to reboot this computer.
Note, there aren’t any packages on this beta CD so you won’t be able to install any software. This is what you do get:

% uname -a
FreeBSD 6.0-BSDINSTALLER-BETA-1 FreeBSD 6.0-BSDINSTALLER-BETA-1 #0: Wed
July 27 10:36:09 UTC 2005 root@x64.samsco.home:/usr/obj/usr/src/sys/GENERIC
i386

Attended David Litchfield’s talk– All New 0-day– this morning. His talk was split into two mini talks, the first covering what he calls SQL Injection and Data Mining Through Inference. The idea is an offshoot of SQL timing attacks, but uses boolean logic to return arbitrary, but meaningful, errors, from the injection attack. By forcing predictable, boolean errors, or really any type of output, he is able to reveal the actual binary information returned by the database.

The second part of his talk, Oracle Patching and Knowing If You Are Not, focused on techniques for determining whether vendor patches actually fix what they claim to. While it focused on Oracle, I’m sure these techniques could be easily applied to other applications/platforms. His slides and a whitepaper are forthcoming.

Dan Kaminsky covered a wide range of topics. First he showed an example of two web pages that yield the same MD5 sum– already on his site.

Next, he explained some issues with IP fragmentation, illustrating his ability to exploit the disparity between IDS and host timeouts to force reassembly of two distinct (one noise and one an attack) fragmented payloads using the same fragmented data stream.

He went on to explain that he had been given access to a really large pipe on which he is able to conduct Internet-scale research. His first target: trust relationships in the DNS infrastructure, especially noting the reflexiveness involved in PTR lookups as well as the arbitrary, but predictable, UDP ports left open by client queries. He also alluded to, but didn’t disclose, some major issues for concern that he has detected.

After that, he illustrated a rapid network mapping project in which he was able to incorporate real-time traceroute data into a Internet map using the Boost Graphics Library. He didn’t go into a lot of detail on how the visualizations were created, but in my quick assessment, his prototype already seemed worlds ahead of the “mapping” features that most SIM (event correlation vendors) tout. It was hard to tell if the BGL was largely to do with it, and not necessarily the data set.ß

He ended with a demo of a 64KB/s video feed tunneled through DNS– Lord of the Dance/Sith from Adult Swim.

Gilman Louie’s keynote was a bit muddled at times, and it was really a general reiteration of the security, both in IT, and in general, is broken because it is ineffective and inefficient. The same thing has been said elsewhere. Users are constantly encumbered by these ineffective/inefficient techniques and so one of a few things happens:

1. They circumvent the security feature– this is pretty obvious. He used an example of a malicious email that rewards the end user with “dancing bears”, noting that most users will do whatever they need to join in the shared experience of the dancing bears reward.

2. They become so paranoid that they deny legitimate access to data– think of desktop user firewalls where users are given a choice to deny or allow access to programs on the fly. If the user denies a valid request for data, the result is probably a help desk call to revert the policy and recreate the event, and in turn the rule. In the meantime, you have a user at a potentially open location recounting access control information to the help desk, but also to a multitude of folks in an airport, hotel lobby, etc.

3. The user simply won’t use the technology in either a secure or insecure way. He used the personal example of PKI and email (meaning key ring management and handling encrypted email), explaining that he generally ignored encrypted email until an out of band event caused him to pay attention to the data contained in it. For example, a phone call or coworker screaming at him in the hallway.

He tied this breakdown in security mechanism to the concept of the OODA-loop (observe-orient-decide-act)– the user needs to optimize this cycle, but bad security policies, decisions, and software interfere with this optimization.

All of this boiled down to his notion that users need to share good, reliable data in an efficient way–”information superiority”–, and often security interferes with this, usually at the expense of the end goal– making money, fighting terrorism, seeing the dancing bears.

In general, an okay keynote, once it got moving. He touched a bit on DRM and the need to index everything (i.e. search engines) whether it was useful or not.

I’ll update more throughout the day, including some notes on David Litchfield’s and Dan Kaminsky’s talks.

Anyone else out here?

Related link: http://conferences.oreillynet.com/os2005/

I’m excited that there will be a BSD booth at this year’s OSCON. Even more so that I’ll be helping to man it and this will be my first trip to the West coast and my first OSCON.

There’s some funny quirks that come with being involved in Open Source. One of these is the fact that you can work very closely, often for hours at a time on a near daily basis, with people you’ve never met in person. Conventions are a great way to address this and give the opportunity to put faces and personalities to email addresses and IM accounts. They also give the opportunity to talk one-on-one; this is the primary reason why I’m the first to volunteer whenever a booth or registration desk needs to be manned.

I’d like to see all of the BSD projects represented at the booth. So far, FreeBSD and PC-BSD are well represented and will have lots of promotional material. We could use some more volunteers representing NetBSD and OpenBSD as we only have one volunteer so far from each project. We haven’t heard back from Dragonfly BSD yet. I’ll also be handing out information about BSD Certification and the flyer for BSD Success Stories.

If anyone is interested in putting in a few hours of booth time or can bring copies of flyers, CDs or other promotional material, please drop me an email to make arrangements. Also let me know if you’re aware of any other BSD related projects that may be interested.

If you’re attending OSCON or are in the Portland, Oregon area on Aug. 3 or 4, be sure to stop by the BSD booth and say hi. It will be in the Exhibit Hall which is free of charge.

Related link: http://lists.freebsd.org/pipermail/freebsd-stable/2005-July/016958.html

I learned through Richard Bejtlich’s blog that the first beta for FreeBSD 6.0 was out.

According to the announcement, a cvsup tag was available so I decided to try it out on my laptop system.

Downloading the new source was a simple matter of changing one line in my custom cvsup file:

*default tag=RELENG_5_4

to

*default tag=RELENG_6

I then ran cvsup and waited an hour or so as a whole bunch of src was downloaded.

Before starting the upgrade, I read through /usr/src/UPDATING to see if there were any gotchas to be aware of. It had this to say:

“NOTE TO PEOPLE WHO THINK THAT FreeBSD 6.x IS SLOW:

FreeBSD 6.x has many debugging features turned on, in both the kernel and userland. These features attempt to detect incorrect use of system primitives, and encourage loud failure through extra sanity checking and fail stop semantics. They also substantially impact system performance. If you want to do performance measurement, benchmarking, and optimization, you’ll want to turn them off. This includes various WITNESS-related kernel options, INVARIANTS, malloc debugging flags in userland, and various verbose features in the kernel. Many developers choose to disable these features on build machines to maximize performance.”

I made a mental note to watch for these as I always create a custom kernel once the upgrade is successful.

To upgrade:

# cd /usr/src
# make buildworld
# make buildkernel
# make installkernel
# make installworld

Then the moment of truth:

# reboot

I watched the startup messages and everything looked good hardware-wise. I was pleasantly surprised at the verbosity of the new DHCP client as I like to see REQUESTS, ACKS and NACKS go by–makes troubleshooting much easier:

Starting dhclient
fxp0: link state changed to UP
no such user: _dhcp, falling back to “nobody”
DCHPREQUEST on fxp0 to 255.255.255.255 port 67
DHCPACK from 192.168.2.100
bound to 192.168.2.71 — renewal in 172800 seconds.

I did a quick poke about and everything still seemed to work: X, sound, pointer, etc. Now it was time for a custom kernel. I removed the debugging stuff:

makeoptions DEBUG=-g # Build kernel with gdb(1)
# Debugging for use in -current
options KDB
options DDB
options GDB
options INVARIANTS
options INVARIANT_SUPPORT
options WITNESS
options WITNESS_SKIPSPIN

I then went through the config file and made my usual customizations. Once I saved the edits:

# cd /usr/src
# make buildkernel KERNCONF=CUSTOM
# make installkernel KERNCONF=CUSTOM
# reboot

My laptop now has this uname -a:

FreeBSD laptop.domain.org 6.0-BETA1 FreeBSD 6.0-BETA1 #1: Sun Jul 17 09:06:44 EDT 2005 root@laptop.domain.org:/usr/obj/usr/src/sys/CUSTOM i386

So far, I haven’t encountered any problems with the new OS and was quite pleased at the flawlessness of the upgrade. A couple things to note if you’re going to take the plunge:

- do it on a test system after backing up your data; this is a beta, after all

- my laptop was fully in sync with 5.4; I don’t know what will happen if you upgrade from a version prior to that

I like Colin Percival’s quote on whether or not to use this beta:

“If I was deploying a new server today, I’d install FreeBSD 5.4. If I were planning on installing a new server next month, I’d install FreeBSD 6.0-BETA-whatever-number-we’re-up-to-by-then.”

Finally, Scott Robbins of NYCBug pointed me to a pr
which some may find helpful. It gives the instructions for disabling malloc debugging and mentions the new “make cleanworld” target. Thanks, Scott, for pointing this out.

Related link: http://pma400.com

I recently picked up an Archos PMA430, an all-in-one portable media player that combines their old AV400 series that can play and record both MP3s and DivX, display photos, and output to a TV using an included wireless remote, with a touchscreen PDA running Qtopia. Read below to find out what I’ve discovered so far, including the hackability of the Linux OS running underneath.

About a year ago I was in the market for my first hard-drive-based mp3 player. I wanted to carry all my songs around with me, and my MP3 CD player just wasn’t cutting it. After doing a lot of research, it came down to a choice between a 40Gb 3rd Gen iPod, and an Archos AV340. Both were about the same price, with the AV340 being a bit cheaper, both had 40Gb of storage, and both had advantages and disadvantages. In the end, the fact that the AV340 had about five times the featureset of the iPod won out over the fact that the iPod was thinner and had the nifty wheel interface. Well, that and I’ve never really gotten into the whole whiter-than-white color scheme.

I’ve always been someone who has liked single devices that can do many things (Swiss Army knives, Perl, Knoppix). Of course, some of my iPod-apologist friends chided me for going with the Archos: “But the iPod does one thing and does it well–play mp3s” they said (while they checked their addressbook, hooked up a microphone, looked at color photos, and installed Linux on their iPod). The iPod is fine, but my AV340 fit in well with this multitool philosophy. I could listen to mp3s in the car, go home and hook it up to the TV and watch a movie (or even record a movie to it), and when I was on vacation I could unload my digital photos using its CF adapter without having to carry around a laptop. Plus there was the fact that I didn’t have to jump through any hoops or install anything to add files to the Archos under Linux–plug it in to a USB port and copy mp3s or music to their respective directory.

My AV340 was great, but when I saw the PMA430 (more tech specs here), I was ready for an upgrade. It had all of the features of my AV340, but it added all of the features of a PDA–a touchscreen, all the PIM tools, plus integrated wireless so I could browse the web on its included Opera browser. Plus it includes a USB host port so you can not only use it as a USB hard drive, you can plug other USB devices into it like cameras, other USB hard drives (even an iPod), USB keyboards, and even USB network and bluetooth adapters. All this and it was much thinner than my AV340. Then I found out it used Qtopia and ran Linux under the hood (like the Zaurus) and I was sold.

Improvements

My expectations for the PMA430 were pretty high, but so far it has met and in some cases exceeded them. For one, the interface is much improved. The arrow keys, OK, and close buttons are situated where it is easy to navigate through them, and what used to be three identical general-purpose function keys are now individual buttons that perform special functions such as tab through open windows, open a window’s menu, or go to the application window. Plus with the addition of a touchscreen you can also do all of these functions with the included stylus or even with your finger. Like with the AV340, you hold down the close button for 3 seconds to power it off, and the OK button to power it on, only now it actually suspends instead of completely powering down so it starts up almost immediately and remembers state so any open applications are still open. Also, the PMA430 can multitask, so I can browse the web, IRC, or check my email while I am listening to MP3s. The new docking station is really nice as well–all of the cables are integrated so it’s easy to load a video on the PMA: pop it on the docking station, connect a single cable, and start playback on my television with the included wireless remote.

Beyond the initial interface improvements, the new featureset is very impressive. Now I can connect to the wireless network at home and browse the web, IRC, ssh to another machine using the terminal app, and even sync my data all over the connection. I also recently bought a generic RTL8150 chipset USB adapter for $12 so I can connect to regular wired networks (It sure beats the $40 Archos wants for their network adapter). You can also use the integrated IrDA port to use the GPRS connection on your cellphone. In many ways the PMA430 is in a different league of competition than standard MP3 players or even MP3 and video players. Probably the closest device in terms of features is the Palm Lifedrive, which also adds bluetooth and an SD slot, but only has 4Gb of space to the PMA’s 30Gb.

It has been some time since I’ve actually carried around a PDA with me (my Palm III is collecting dust in a drawer) so I haven’t used the PDA functionality too much yet, although I did just start adding to the contact list. The PIM apps seem pretty standard–no frills, just the general stuff you would expect. I was able to grab the Linux version of Qtopia Desktop and sync with the PMA430 over the wireless connection with no problems.

Linux under the hood

As I mentioned before, the PMA430 uses the Qtopia environment like a Zaurus, and if you open up the terminal application, you can throw standard Linux commands at it. Because it uses Qtopia, there has been a lot of talk about the potential to run Zaurus apps on it. Kevin Boone’s PMA430 FAQ goes over the specifics, but basically a lot of the newer apps (written to work on both the old Zauruses as well as the clamshells) might possibly work without any additional work, although many require that you at least repackage them to change some of the directories they use (PMA430 prefers /opt for packages instead of /usr). Other applications, especially those with hard-coded portrait dimensions, might require some extra hacking and recompilation to get working. Kevin also covers some of that on his page, and Archos has also recently released a software development kit for the PMA430 so you can write your own applications. A few programs have already been written including a a streaming audio player, and there are a few sites that have a number of ported Zaurus applications on it such as http://www.kevinboone.com/pma430_conversions.html and http://www.jbmm.fr/index.php?ind=downloads&op=section_view&idev=1.

The Archos SDK also includes the source and config to their 2.4.19 Linux kernel, which means you have the potential to add extra hardware support to the device (particularly for usb devices) if it doesn’t include it. Already there are packages around to add USB serial adapter and GPS device support. The important thing to keep in mind is that the Linux environment on the PMA isn’t exactly like a Zaurus. The PMA uses a few loopback filesystems with a number of symlinks for its root filesystem and mounts the general storage under /media. When the device reboots, anything that isn’t under /media (or isn’t a program installed in the /opt loopback filesystem) gets erased. This means you can’t just copy files wherever you want, you need to do a bit of planning and (ideally) package up your files first.

Nitpicks

The PMA430 is great and I can’t say enough good things about it. There are a few little nitpicks however. For one, it can be quite a pain to use the integrated touchscreen keyboard with the terminal. The keyboard uses up a bit less than half the screen, and as this is a landscape orientation most apps have to shrink pretty small to fit. Unfortunately the terminal app doesn’t really attempt to resize at all, so if the cursor is at the bottom of the screen you have to type blind and then minimize the keyboard to see the contents of the terminal. I actually just ordered a folding USB keyboard like you might use with a PDA, so this will become less of an issue, and the resizing problem only seems to really affect the terminal. Also, I have had a few crashes here and there. Granted I have been busy installing all sorts of random Zaurus apps to see if they will work, so that might have something to do with it. Basically I had to hit the hardware reset key and everything was back to normal. Also one time I had to reinitialize the internal MP3 database it keeps, as it got corrupted, but that was a simple matter of deleting a file and telling it to rescan. Any other issues I’ve had with the PMA430 are basically program crashes on Zaurus programs I’ve tried to get running that haven’t been fully ported yet.

Conclusion

What can I say? I really am impressed with the PMA430. It does all of the stuff my old Archos did in a smaller package, adds a ton of new features, plus there is a great potential for new functionality on the horizon. There are a number of communities behind the project including archosmultimedia and archos_PMA400 Yahoo! Groups, and more apps are being ported or written every day. If you like hackable feature-rich devices, I recommend you give this a try.

I’m used to dealing with SNMP. I’ve worked with three or four monitoring and graphing applications in the past, all of them being open source. I’ve mapped IP addresses to interface indexes, and back again. I’ve created RRD files from the command line, and I learned to transmit my own custom SNMP traps to management servers.

I can recognise a lot of OIDs by sight nowadays. I recognize them, because I am usually forced to type them into those open source network managing applications. Sometimes it works, sometimes it doesn’t.

Almost every open source graphing program out there knows how to do ifInOctets and ifOutOctets. MRTG set the standard on that one. With the availability of RRDTool, we have a flooded market of tools that graph all sorts of network statistics just like MRTG did.

But what I don’t see are tools that can handle more complex SNMP polling situations. When the open source programs fail, I find myself being forced to write my own polling program (usually in Perl) that runs independently from the main polling program.

I wrote a program to monitor an SNMP device, because OpenNMS couldn’t deal with a broken SNMP agent implementation. I wrote a program to monitor a Network Appliance, because MRTG couldn’t easily re-create the “Operations Per Second” statistic. The list goes on and on.

What I need is one single polling program that can accomplish all of these tasks.

1. Properly merge high and low 32 bit counters.
2. Read a MIB file.
3. Perform mathematical operations on retrived values before saving.
4. Successfully navigate a SNMP table index (an index that uses a string of sub-identifiers).

If there’s one program out there that can easily do all this, I haven’t found it yet. But then again, there are so many programs out there, I may have missed one.

Give me your feedback. Find me the program that does the job.