My first experience collecting commodity prices for a trade publication was more than 30 years ago.
A thick skin has always been one of the requirements for such work. I still wince whenever I thumb through my copy of a 1977 transcript of a House of Representatives subcommittee grilling of the publisher of a livestock newsletter on his procedures for gathering prices. One of the topics from that session still resonates with me The more a price table influences the market, the thinner the "free market" information that's truly independent of the published numbers. Too much popularity degrades the information.
In the late 1970s, I became statistics editor of a newly formed weekly that covered industrial fuel and power. I got to watch the creation of the Energy Information Administration (EIA). Now I could learn how specialists with clout and serious training handled price disclosure, I felt.
My favorite lesson from that period occurred when I was updating a table on refineries' selling prices for non-diesel No. 2 oil in Pennsylvania. How could prices be soaring for August and September in a year when markets were soft? An EIA staffer understood my bafflement. "Bimodal distribution," he explained. Two sets of prices contributed to the average. Most of the year, the refineries sold almost entirely to large customers. In late summer, however, quite a few mom-and-pop heating oil marketers stopped by to purchase their autumn and early winter requirements at summer prices, briefly changing the refineries' customer mix. Those small buyers, needing modest quantities once or twice a year, were on a costlier rate card than the big guys, briefly pushing up the average.
That case exemplifies another of those generically tricky issues. How much editing should be done to disregard transactions or postings that are genuine but "don't really count" from the standpoint of most readers wanting the data?
When I began my second stint at AMM in 2000, I realized that price trends for several types of steel imports are tracked by a carefully designed survey done by the Bureau of Labor Statistics. (Accuracy is important here because economists often need to quantify the dollar volume of imports and exports with the price changes filtered out.)
But there were implausible shifts in the BLS steel price indexes that smelled wrong. It turns out that respondents, such as steel importers, are placed in the sample for rigid four-year stints. Domestic steel mills were successful in 2001 and 2002 in getting importer-specific anti-dumping duties imposed on many types of imported steel. An affected importer, and its foreign supplier, might lose nearly all their customers in a niche, with obvious disruption of pricing. Alas, BLS wasn't allowed to evict such a firm, or minimize its weight in the formula, until the stipulated four-year span came to an end. The result Crummy data, despite the elegance of BLS' methodology in other respects.
The latest wrinkle affecting price reporting is a move by the Commodity Futures Trading Commission (CFTC), which is trying to prosecute an unnamed natural gas trader that allegedly misled a newsletter about prices, without actually lying, with the goal of making improper profits from futures swaps. Only one of the court documents is public at this point, but it makes curious reading. The CFTC acknowledges explicitly it isn't alleging that "false trade data" reached the publication; rather, the trader allegedly was "infecting the index with manipulated, i.e. 'artificial,' prices in an effort to skew that index."
My translation The reported deals did actually occur, but they were corruptly motivated to fool other traders or else they were accompanied by other deals that weren't reported to the publication, Inside FERC.
Fining offenders $1 million plus for those sorts of sleaze would go a long way toward ensuring the accuracy of pricing journalism. I haven't yet chosen to endorse that solution.