What's Up DOCumentation Robelle Consulting Ltd. 8648 Armstrong Rd., R.R.#6 Langley, B.C. Canada V3A 4P9 Telephone: (604) 888-3666 Telex: 04-352848 Date: February 12, 1988 From: Robert M. Green, President David J. Greer, Research & Development Michael C. Shumko, Customer Support To: Users of Robelle Software Re: News of the HP 3000, 1988 #1 What You Will Find in This News Memo: Spectrum Finally Leaves the Nest News Tidbits The Top Ten Earthquake Concerns of HP 3000 DP Managers Technical Tips About Robelle Squeezing the Last Bit From Your HP 3000 Part Four: Use MR NOBUF Tools and Optimize Block Sizes Robelle Products: Problems, Solutions, and Suggestions Spectrum Finally Leaves the Nest Hewlett-Packard has officially released the 930 and 950 Spectrum computers, finally abandoning the protection of non-disclosure agreements. We have heard from several sources that the 930 and 950 attained Manufacturing Release during the month of January. This means that people who received "Control Shipment" Spectrums can now put them into production and let outsiders use them. You no longer need to sign any restrictive agreements to get a 930/950. It also means that we users can now compare notes on what the MPE XL systems are good for. Interestingly, we didn't hear about the Manufacturing Release (MR) of the Spectrum from Hewlett-Packard itself. As far as we can determine, HP kept this event very quiet - no press conferences or splashes of publicity. Even some HP people in Cupertino were not aware that MR had occurred. Just because the 930 and 950 are released does not automatically guarantee that you can get one. Given the huge backlog of orders that HP has, it can still make "controlled shipments" for a while, picking sites whose expectations match the state of the machines. The first supported release of MPE XL is A.01.00. If you have a 930 or 950 and our SUPRTOOL, you might want to help fellow users get a better handle on the capabilities of these new systems by running some performance comparisons. We have outlined some interesting tests below. If you get a chance, try them on your machines and send the results to us. We will compile the performance figures and publish them in a future news memo. Spectrum Performance Tests Select a large IMAGE dataset from a database that you have on both a Classic 3000 and a new MPE XL 3000. Then use SUPRTOOL to execute two typical tasks on this dataset. First on the Classic 3000, then on the MPE XL system, using the standard SUPRTOOL (compatibility mode); then use a copy of SUPRTOOL that has been translated by the OCT (object code translator). Try to perform the six tests on an unloaded system. :run suprtool >set stat on {prints run-time statistics} >base yourDB >:comment Test #1: Typical Task With Sort >get yourset >extract field1,field2,field3 >if field1 > "xxxx" ... >output xxxx1 >sort field1 >xeq >:comment Test #2: Same Task Without Sort >get yourset >extract field1,field2,field3 >if field1 > "xxxx" ... >output xxxx2 >xeq Which Classic 3000 did you use (i.e., 68, 70, etc.)? __________ Which MPE XL machine did you use (i.e., 930 or 950)?__________ What was the MPE/V version number? __________ What was the MPE XL version number? __________ What was the SUPRTOOL version number? __________ How many on-line users have you run under MPE XL? __________ How would you describe the performance? _____________________ ______________________________________________________ The SET STAT ON reports from SUPRTOOL will give you the elapsed and CPU times for each test. Thanks in advance for your cooperation. News Tidbits Series 970 Dead? We have been hearing hints from HP over the last year about a new Spectrum processor with twice the power of a 950. The December 1987 issue of Datamation carried the following rumor: "HP Pulls The Plug. Hewlett-Packard, after telling its largest customers over a period of months about a new, high-performance ECL-based version of its Spectrum minicomputer, has pulled the plug on the development project, sources say. HP chief operating officer Dean O. Morton, who announced the decision internally, reportedly told the 100-member ECL development team that the project was being killed to save on research and development expenses. Sources say the company already has poured over $20 million into the project, which had been run out of HP Labs in Palo Alto, had been under fire from rival developers at HP's growing Cupertino systems group for some time. As previously reported [in Datamation], that group plans to build follow-on Spectrum systems using circuit technology that HP will get under a secret deal with IBM." HP Plans. Hewlett-Packard managers fielded some interesting questions at the recent Scottish users conference, including this one: Q. The HP 3000 RISC systems are in the top range; when will HP develop its low end? A. "We intend to develop both the high and low ends in 1988. The 925 system will replace the 52/58 as a mid-range machine that will span up to the Series 70 but not replace it. There will also be 950 field upgrades in `88 as well as a new RISC-micro based on MPE V architecture." [??!!] SUPRTOOL Success. In the Las Vegas proceedings, there is a good article by Mark Tolbert who works for the Customer Escalation Center of Hewlett-Packard. The following quote is from "Evolving Performance Guidelines - An '87 Update": "For database batch processes, using products such as SUPRTOOL and ASK can significantly reduce serial access times against large datasets. In one case I worked on, a customer was able to reduce an 11 hour batch report job that needed to run overnight, to 3 hours using SUPRTOOL." Spectrum Quote from the SCRUG newsletter: "The SPECTRUM has evolved from an abstract theoretical concept to an L.A. commuter (they go real slow) to a nuclear submarine (they stay down for months at a time) to a very nice high-end machine." Alfredo to Speak to GLUG. Alfredo Rego will be presenting a new talk entitled "Adager Internals" to the GLUG on March 8, 1988. He will discuss the technology behind the performance improvements in the new version of Adager. The meeting will be at the Beverly Hills Ramada Hotel (not at the Russian Tearoom any more), includes dinner, and costs $20. For reservations, call VESOFT at (213) 282-0420. QEDIT and SUPRTOOL at the SEMC. Paul Houtz at Hewlett-Packard's SEMC (Software Evaluation and Migration Center) recently wrote an application system to keep track of controlled Spectrum shipments. The SEMC evaluated each potential customer to see that they did not have any unreasonable expectations and to make sure that they didn't want any software that is not available on MPE XL 1.0. What's interesting is that Paul wrote the entire application using QEDIT and SUPRTOOL! Paul left lots of room for free-format text. When an engineer wants to modify the notes on a customer, he uses a series of UDCs and /Use files that extract the data, invoke QEDIT to modify the text, then invoke SUPRTOOL to delete and replace the text. Paul wrote it as a prototype, but HP's CSY division liked it so much that they took it away and started using it. MPE XL Look-Alikes. The MPE XL command interface has some nice new features, such as variables, that would be handy to have on MPE V. Eugene Volokh of VESOFT reports that his newest version of MPEX supports the MPE XL command interface, including variables, but is not plug-compatible with MPE XL because of differences in error-handling. Another vendor, Taurus, has announced a product called Chameleon that claims to provide a complete MPE XL emulation. That is, you should be able to move an MPE XL job stream to MPE V and execute everything without change. We would be interested to hear from any users who have tried Chameleon. In June, Chameleon is projected to have an intrinsic interface (hpicommand). VESOFT doesn't have this feature (and neither does Chameleon yet). Chameleon will probably appeal to larger companies worried about conversion to MPE XL. Chameleon sells for $5000 US; contact Taurus at (415) 853-6893. Beyond RISC. You can now buy copies of the new book "Beyond RISC" that looks into the implementation and implications of the HP 930 and 950 computers. The book was written by a highly qualified group of non-HP consultants who have extensive contract experience with HP on the Spectrum project. Pricing of 1-4 copies is $45 US each, 5-49 is $40, 50-99 is $35 and 100-999 is $30. Send purchase order to Software Research Northwest at 17710 100th Avenue SW, Vashon Island, WA 98070, telephone: (206) 463-3030. The Top Ten Earthquake Concerns of HP 3000 DP Managers People in Southern California worry about earthquakes. An example is this excerpt from the SCRUG newsletter which gives a tongue-in-cheek list of earthquake disaster worries at HP 3000 sites: One: Disks will become fragmented -- physically. Two: Location-sensitive tape management system will collapse when tapes fall on floor. Three: There will not be enough doorways in the computer room. Four: The raised floor will break and the DP Manager will fall through and will be strangled by wires. Five: A blunt object will fall on the DP Manager's head and cause the loss of the MANAGER.SYS password. Six: The backup site in Santa Monica will fall into the ocean. Seven: The only write ring in the shop will fall into a crack in the earth, never to be found. Eight: The Caps Lock key on the console keyboard will break, forcing the DP Manager to use lower-case. Nine: The tape drive will have to be cleaned again. Ten: The uptime van is in the shop for a tune-up. Technical Tips Technical Oddity. Recently, Vladimir Volokh pointed out the following MPE oddity. Enter these three MPE commands: :build xa;rec=-3,,,ascii :build xb;rec=-3,,,binary :listf x?,2 {? means a single alpha-numeric} ACCOUNT= GREEN GROUP= MIKE FILENAME ----------LOGICAL RECORD----------- ----SPACE---- SIZE TYP EOF LIMIT R/B SECTORS X MX XA 3B FA 0 1023 85 4 1 7 XB 2W FB 0 1023 64 3 1 6 There is a serious difference between the files -- what is it? As a result of this difference, the files do not use the same amount of disc space. Can you explain what is happening? KLA/EXPRESS-3000. A company called KLA & Associates sells a product that adjusts the dispatcher queues dynamically to make maximum use of all system resources. We heard about it from Pete Hansen at Hughes via GLUG. He says that everyone at Hughes who has tried it has reported amazing improvements in performance. Another user who tried it said that it appeared to work well, but that you have to tell it which processes should get top priority. This was a problem because, of course, each user thought that his own application was more important than the others', and that the tool should favor his over the rest. They "solved" the dilemma by not getting KLA-EXPRESS. For more information, contact KLA & Associates at (813) 784-5976. More Fun With File Commands. Here's a little-known feature of the CREATEPROCESS intrinsic, which is used by QEDIT's RUN command. /:file x=listeq5.pub.sys /:run *x I still haven't figured out a practical use for this. Maybe in a UDC where one program decides what the next program will be and sets the file command accordingly. A recent SuperGroup magazine described how this could be used to allow COBEDIT to use QEDIT instead of EDITOR for changing copylib entries. About Robelle Rug Talks in March. On March 22, 1988 Robert Green will address the NEORUG (Cleveland, Ohio); on March 23 he will address users in Detroit, Michigan; and on March 24 he will speak in Indianapolis. Whew! Bob's topics will be "How to Squeeze the Last Bit From Your HP 3000" and "The Unoffice, Part II, More on Working at Home". Robelle in Sweden. Robelle will be present in GÖteborg in force. The meeting runs from May 30 to June 3 and promises to be exciting and entertaining. This is the first international conference since HP lifted non-disclosure on the Spectrum. Mike Shumko of Robelle will be present to answer support questions in our booth and to present our talk "Squeezing the Last Bit". Marie Froese will be "manning" our booth and is ready to field any sales questions. Robert Green will be presenting two papers: "The Spectrum Instruction Set" and "The Unoffice, Part II". We hope to meet as many European users of Robelle software as we can, so please come forward and introduce yourselves. Squeezing the Last Bit From Your HP 3000 Part Four: Use MR NOBUF Tools and Optimize Block Sizes We have done an informal survey of large HP shops to find out how the successful ones avoid topping out the HP 3000 line. One of the most powerful ideas mentioned was to: Use MR NOBUF Tools and Optimum Block Size. Problem. One of the most common destroyers of system performance is the notorious serial scan. When you copy an enormous file, or reorganize a KSAM file, or select 100 records to report with QUIZ by reading every entry in a million-record dataset, you are bogging down the computer. The default methods of doing a serial scan are extremely inefficient on the HP 3000. Solution. One of the most impressive ways to speed up serial I/O is to use MR NOBUF (multi-record non-buffered, not Mister Nobuf). You can write your own code to take advantage of MR NOBUF access if you're careful, but you don't need to - you can purchase tools that do it for you. Popular tools using MR NOBUF access are HP's DSCOPY (you can use DSCOPY for copying files to the same system), HP's COPYCAT for file copying and backup, MPEX's FCOPY/FAST, and Tymlabs' COPYRITE for file copying and duplication (powerful for KSAM users). Robelle's SUPRTOOL does MR NOBUF serial file access for IMAGE datasets (and any other file type), and Running-Mate replaces serial dataset reads in applications. The Power of MR NOBUF We got a call a while ago from a fellow who didn't even know he had SUPRTOOL on his system, because it came bundled with another package he had bought. He found it AND the documentation on his system, so he started using it. He had a QUIZ job that normally took two hours to run, cruising through a huge database. A total novice, he followed the instructions in the manual to get SUPRTOOL to front-end his QUIZ report. The total time for this daily job went from two hours down to 15 minutes. One of the shops we interviewed still uses a service bureau for some big accounting merges in IBM batch. They're considering using a Spectrum instead, if it's big enough. They reduced the number of service bureaus they needed from four to one by bringing jobs in-house with their own machines and finding packages, such as mailing-list software, that they could front-end with SUPRTOOL. Turbo Resources use their HP 3000 to bill their credit card customers. At month-end, they had a batch program that generated 1,000,000 disc I/Os reading a 90-record control file. Sixty of the records were unnecessary; after reblocking the file, they were able to read it in one disc I/O. They now keep the control information in a table in memory, reducing one million disc I/Os to one. Block Sizes The default blocking factors (number of records per physical disc block) is usually wrong. For big batch disc files, the maximum block size is now about 14K words (REC=14,336), while the default is still the smallest block that will fit. The bigger the block, the faster the programs will run. For IMAGE databases, the default block size is 512 words, as it has been since 1974. Many people we contacted in our survey were using 1024 words or more. Robelle Products: Problems, Solutions, and Suggestions SUPRTOOL Version 2.9 and 2.9.1 System Configuration for SUPRTOOL 3.0. When SUPRTOOL version 3.0 is released later this year, it will require a larger maximum extra data segment size. Previous versions required 8196 words, but this new version requires 16,384 words. Before installing SUPRTOOL, you should check your system configuration with the :SYSDUMP command. The maximum extra data segment size is found under "SEGMENT LIMIT CHANGES". If this value is less than 16,384, you must create a new COLDLOAD tape with the limit increase, then restart your HP 3000 with the COLDLOAD option. QEDIT Version 3.6 MpexHooking QEDIT. Here are the instructions for MpexHooking QEDIT, for your information: :run hookprog.pub.vesoft;parm=1;info="qedit.pub.robelle" HOOKPROG Version 0.1...02 (VESOFT, Inc. (C) 1987) Allow % (Y/N)? y Silent string (CR is good default)? q Minimum input length to keep (default 1)? 10 Maximum input length to keep (default 32767)? 300 Patch READ (Y/N)? n Patch READX (Y/N)? y Patch FREAD and FOPEN (Y/N)? n Patch LOADPROC (for VEMODIFY) (Y/N)? y END OF PROGRAM Tim Joseph at VESOFT tells us that the latest version of HOOKPROG recognizes QEDIT and does the proper things without all this fooling around. Few people have this new version yet. QEDIT Installation. The QEDITJ1A job stream installs the HP compilers in the Q.ROBELLE group (isolated method). Unfortunately, this job adjusts the capabilities of the group incorrectly. The result is that some versions of QEDIT and Fortran 77 will not run: "illegal capability". To correct the Q group, use these commands: :hello mgr.robelle :altgroup q;cap=ia,ba,ph,ds,pm,mr You should also change the job, so it will work properly the next time you stream it. Native-Mode Compilers on MPE XL. There are some problems using the NM compilers from QEDIT 3.6. First, the UDCs that we provided in Udcxl.Catalog.Robelle are now out of date, since HP has changed some program and file names. Second, the QINPUT option of QEDIT's :RUN command, which these UDCs depend heavily upon, has a serious bug: it doesn't write the sequence numbers properly. This makes it impossible to use the NM COBOL compiler without first doing a KEEP. This is fixed in QEDIT version 3.6.2. Converting Files to QEDIT format. As with all things, there is a right way and a wrong way to convert a group of files to QEDIT format. First, here is a wrong way: :run mpex.pub.vesoft %qedit @.source,shut * {DON'T DO THIS!} %exit This command instructs MPEX to invoke QEDIT, and for each file in the group, to SHUT it with the same name as TEXTed. Seems simple enough. But the flaw is in the assumption that MPEX will generate a sequence of /text filename; shut * commands. In fact, that is exactly what MPEX will do for any files that are not already QEDIT files. But MPEX, trying hard to optimize its performance, issues an OPEN instead of a TEXT when it encounters a file that is already in QEDIT format. For QEDIT files, then, the command generated by MPEX is /open filename; shut *. This has the undesirable effect of trying to shut the file with whatever name was used the last time that file was TEXTed or KEEPed. The last text/keep name is retained in the QEDIT user label from one session to the next, and may be months or years old! This could attempt to rename the file as something totally unpredictable, such as a file in another group or account, if that's where this file originally came from. If you're lucky, the shut will fail for some reason, such as attempting to rename across accounts, or invalid group name. The file would then retain its old name. It's also possible that the file has never been texted or kept, in which case the SHUT * would surely fail. But if you're unlucky, the SHUT * will actually work, renaming your file so that you won't be able to find it, and possibly purging a perfectly good existing file in the process. It's possible that a number of QEDIT files were all originally TEXTed from a single file. In that case each one would be shut to its original name, one after the other, until all you were left with was a single QEDIT file having only the contents of the most recently SHUT file. Wow! These are a lot of undesirable consequences for such an innocent-looking command. I should point out that both MPEX and QEDIT are performing perfectly, doing exactly what you asked them to do. The problem lies in not realizing exactly what you're asking. The HP 3000 is not yet a do-what-I-meant computer, unfortunately. If you feel that you must use MPEX, here is a slightly better method: %qedit @.source(code<>qedit),shut * {Still don't do this} This command omits QEDIT files from the fileset, thus ensuring that QEDIT files won't get renamed into oblivion. I say slightly better, because this still has the effect of converting everything that's not QEDIT into QEDIT. This includes program files, USL files, VPLUS forms files, etc. This is still not good. A better way of using MPEX is the following command: %qedit @.source(code=0),shut * {Do this for SPL, Fortran, etc.} %qedit @.source(code=edtct),shut * {and do this for COBOL} This will convert only files that have no filecode or are COBOL source. This is a lot closer to the desired effect. It's still possible to convert a file that shouldn't be converted (e.g., a job stream or data file), but there's no helping that. By far the best method of converting files to QEDIT format is to use our QCOPY program. It automatically skips over files that are already in QEDIT format, and omits any files that shouldn't be converted, such as program files, etc. The syntax is similar to FCOPY, and it accepts the at-sign wildcard to indicate all files in a group. For example, to convert a group of files to QEDIT format, leaving them with their same names: :run qcopy.qlib.robelle >from=@.source;qedit >exit Or, to convert them, making copies and leaving the originals untouched: >from=@.source;to=@.qsource;qedit Now you know the right way, and a whole bunch of wrong (or less right) ways. Take your pick. N.B. A new version of MPEX, expected to be released by summer '88, will reportedly eliminate the problem by allowing you much more flexibility in specifying the operations to be performed on the qualifying file. It will also allow you to specify (code=0 or edtct) in one pass, an operation which currently requires two MPEX passes. SELECT Version B.07 Just released. New features: :MPE commands at SELECT command prompts; "Okay to exit" question to prevent accidental exit. QLIB Version 4.0 QueryCalc. Some of you who tried Equater! in our latest QLIB were intrigued by the built-in "sales pitch" for QueryCalc. For your information, the price of QueryCalc is $5500 US until February 15, 1988, $7000 US thereafter. Contact AICS in New Mexico directly at (505) 526-6842. TUNER. Version A.00.06 was dropped from the QLIB this month, as HP distributes its own version A.00.10 of TUNER in TELESUP.