Skip to content

Home

Jump to bottom
anjenkin edited this page Sep 1, 2023 · 183 revisions

Release Log

August 8, 2023

Website Updates

  • New Award Filter
    • Added filter option in Advanced Search to allow users to search for only new awards. Selecting the “Show New Awards Only” checkbox will display any prime award whose base transaction action date (the date of the first transaction of a prime award) falls within the selected time period.
    • This functionality is also available via our API endpoints that accept a time period filter by using “date_type”: “new_awards_only”.
  • Added the ability to filter in Advanced Search by Congressional District “90” (Multiple Congressional Districts) in states with more than one congressional district.
  • Added copy clarifying how loan awards are displayed across the Time, Map, and Categories tab in Advanced Search. Loan awards use the subsidy cost rather than the obligated amount to sum up the value of the loan.
  • Linked to Custom Award Data Download from the Advanced Search Download feature to help navigate users that would like to download more than 500,000 records.
  • Updated the COVID-19 Profile Page Award Spending by Recipient map with New Congressional District geography layers.
  • Removed the “Obligated Amount” and “Outlayed Amount” rows from the $ Award Amounts section of the Award Profile page to clarify that loan awards only have “Original Subsidy Cost” and “Face Value of Loan” data.

Download/API Updates:

  • Added State and County FIPS (Federal Information Processing Standards) Codes for Prime Award Place of Performance and Recipient Location to the Subaward download files. The following columns were added:
    • prime_awardee_state_fips_code
      • location in file: immediately before prime_awardee_state_code
    • prime_award_primary_place_of_performance_state_fips_code
      • location in file: immediately before prime_award_primary_place_of_performance_state_code
    • prime_awardee_county_fips_code
      • location in file: immediately before prime_awardee_county_name
    • prime_award_primary_place_of_performance_county_fips_code
      • location in file: immediately after prime_award_primary_place_of_performance_city_name
    • prime_award_primary_place_of_performance_county_name
    • location in file: immediately after prime_award_primary_place_of_performance_county_fips_code

Bug Fixes

  • Fixed a bug that prevented Advanced Search results to be returned when special characters were used in the recipient filter.
  • Fixed a bug that caused the Custom Account Data download files to return multiple rows associated with the same federal account symbol and submission period.
  • Fixed a bug that caused the tooltip in Spending Explorer to not open in tablet or mobile view.
  • Fixed a bug that caused the main site navigation dropdown to not disappear as a user tabs through the main navigation and then to the rest of the page.
  • Fixed a bug that caused Non-Award Spending to appear in the Spending Explorer breakdown of Recipient by Awards.
  • Fixed a bug the caused the “Download the Data” items to be unclickable on the main site navigation.

July 19, 2023

Website Updates

  • New Congressional District Data:

    • Added filter options in Advanced Search to distinguish current congressional districts (based on the 2020 U.S. census and congressional reapportionment) and original congressional districts (based on data received or derived by the USAspending source system Broker). In addition, updated the Map tab to display current congressional district data.

    • Updated the Award Profile pages to display current congressional district data in the Recipient description at the top of the page, and in the Additional Information section at the bottom of the page.

    • Updated the State Profile pages to display current congressional district data in the map for Primary Place of Performance and in the Top 5 table for congressional districts.

    • Added an explanatory article on Congressional District Data to the About the Data resource module.

    • Updated congressional district fields in the following downloads:

      • Advanced Search download
      • Custom Award Data download
      • Award Data Archive download
      • Award Profile download
      • COVID-19 Spending Profile download

      Files generated with the following endpoints will be impacted:

      • /api/v2/download/transactions/
      • /api/v2/download/awards/
      • /api/v2/bulk_download/list_monthly_files/
      • /api/v2/bulk_download/awards/
      • /api/v2/download/contract/
      • /api/v2/download/idv/
      • /api/v2/download/assistance/
      • /api/v2/download/disaster/

      All existing congressional district fields will be removed from these files and replaced with new columns. The values in each column will follow a “XX-##” format (where “XX” represents the state postal code and “##” represents the district number). The new columns names are:

      • prime_award_summary_recipient_cd_original
      • prime_award_summary_recipient_cd_current
      • prime_award_summary_place_of_performance_cd_original
      • prime_award_summary_place_of_performance_cd_current
      • prime_award_transaction_recipient_cd_original
      • prime_award_transaction_recipient_cd_current
      • prime_award_transaction_place_of_performance_cd_original
      • prime_award_transaction_place_of_performance_cd_current
      • subaward_recipient_cd_original
      • subaward_recipient_cd_current
      • subaward_place_of_performance_cd_original
      • subaward_place_of_performance_cd_current Column names with “prime_award_summary” contain prime award summary level congressional district information.

      Column names with “prime_award_transaction” contain prime award transaction level congressional district information.

      Column names with “subaward” contain subaward level congressional district information.

      Column names with “recipient” contain legal entity recipient congressional district information.

      Column names with “place_of_performance” contain primary place of performance congressional district information.

      Column names with “cd_original” contain congressional districts received or derived by the USAspending source system Broker.

      Column names with “cd_current” contain congressional districts derived using the current congressional district map.

  • Award Profiles: Updated the display logic of outlay amounts in the Award Profiles so that blank values are now shown as “--".
  • API: Added a link to the API tutorial video, as well as a link to a new API tutorial slide deck, in the Documentation Index page.

Bug Fixes

  • Fixed a bug that affected the county and congressional district data in the State profiles for U.S. territories. The map for Primary Place of Performance and the Top 5 tables now properly display these location data.

  • Fixed a bug that caused an incorrect URL path when navigating from the COVID-19 Spending Profile to the Agency Submission Statistics page.

June 28, 2023

Bug Fixes

  • Fixed a bug in the Agency Profile’s Status of Funds chart that prevented drilldown and display of data for Treasury Account Symbols that include a slash (“/”).
  • Fixed bugs in the Agency Profile’s Status of Funds chart in order to display proper formatting for hover-state pop-ups, lengthy text headers, and bar chart proportions for negative values.

June 7, 2023

Website Updates

  • Advanced Search:
    • Added a column for “Total Outlays to Date”. This column is placed to the right of the column for “Total Obligations to Date” (previously labeled “Award Amount (Total Award Obligations to Date)”). A column value of “--” indicates that no outlay information is available.
    • Added a Share button to the Advanced Search page. Users have always been able to share the URL for a submitted search to recreate search results—this button makes the share functionality more explicit and noticeable.
    • Added a note and links to explain how the download/transactions and download/awards endpoints allow API users to select columns to include in their download package. This approach is currently the best way for users to customize their downloaded columns in Advanced Search.
  • Agency Profile pages:
    • Added the fifth and final drilldown layer to the Status of Funds chart. The fifth layer shows either Object Class breakdowns if the fourth layer selected Program Activity, or Program Activity breakdowns if the fourth layer selected Object Class.

*** Training Videos:**

*** Downloads**:

  • Standardized the data throughout our award downloads for county FIPS codes and state FIPS codes, affecting both recipient location and place of performance. County FIPS codes now appear in a five-digit format (the first two digits for the state code, and the last three digits for the county code), which is unique for every county in the United States. State FIPS codes will remain in the two-digit format. The following downloads are affected by this change:
    • Contracts_PrimeAwardSummaries and Assistance_PrimeAwardSummaries
      • Found in Advanced Search
    • Contracts_PrimeTransactions and Assistance_PrimeTransactions
      • Found in Advanced Search, Custom Award Data, and Keyword Search
    • Contracts_Full, Contracts_Delta, Assistance_Full, Assistance_Delta
      • Found in Award Data Archive

Bug Fixes

  • Fixed a bug that affected usability of the “Copy Link” option in the Glossary module’s share feature within entries.

May 17, 2023

Website Updates

  • Updated our site navigation based on user feedback to better explain the features and data on USAspending.gov. Certain features have been reorganized (e.g., Spending Explorer and Profile pages have been grouped under a new “Explore the Data” category, and the Keyword Search can now be found as a link within “Search Award Data”). All site navigation features now include a short description.

  • Renamed the “Analyst’s Guide” to the “Federal Spending Guide”, and updated the URL accordingly.

  • Added an additional drilldown layer in the Agency Profile’s Status of Funds chart. Users can now view breakdowns by Program Activity or Object Class within any given the Treasury Account Symbol (TAS).

  • Updated the award transaction-level date filter for Disaster Emergency Fund Code (DEFC) filter selections in Advanced Search. Previously, all results for COVID-19 DEFC filter selections in Advanced Search showed award transactions starting in April 1, 2020, the date when such DEFC tracking began for federal agencies. Now, the date filter is specific to the earliest date of enactment of any supplemental appropriations law applicable to the DEFC value selected. The updated date filter applies to both COVID-19 and Infrastructure DEFC values; it prevents the display of award transactions that could not have been funded by the law associated with the selected DEFC value. Read more about this topic in the updated “Start Date for Disaster Emergency Fund Code (DEFC) Tracking” article in About the Data.

    For reference, below is a list of DEFC values and the earliest date of enactment for their relevant law(s).

    • DEFC “L”: 3/6/2020
    • DEFC “M”: 3/18/2020
    • DEFC “N”: 3/27/2020
    • DEFC “O”: 03/27/2020
    • DEFC “P”: 4/24/2020
    • DEFC “U”: 12/27/2020
    • DEFC “V”: 3/11/2021
    • DEFC “Z”: 11/15/2021
    • DEFC “1”: 11/15/2021

Download Updates

  • Added columns for Total Outlays to award downloads. Where no outlay data has been submitted, the cell in this column will be blank.
    • In Award Summary files (found in Advanced Search download and COVID-19 Profile download), this column is labeled “total_outlayed_amount”.
    • In Award Transaction files (found in Advanced Search download, Award Data Archive download, Award Profile download, and Custom Award Data download), this column is labeled “total_outlayed_amount_for_overall_award”.
    • In Subaward files (found in Advanced Search download, Award Profile download, COVID-19 Profile download, and Custom Award Data download), this column is labeled “prime_award_total_outlayed_amount”.

April 26, 2023

Download Updates

  • Added columns for obligation and outlay amounts funded by the Infrastructure Investment and Jobs Act (IIJA, also known as the Bipartisan Infrastructure Law) to the COVID-19 Spending Profile download (Prime Award Summary files and Subaward files). Please see the release notes from April 4, 2023 for other additions of IIJA data to USAspending downloads.
  • In order to facilitate faster downloads in Custom Account Data Download, our File C (Account Breakdown by Award) file has been split into three buckets: contract awards (with linked awards between File C and File D1), financial assistance awards (with linked awards between File C and File D2), and unlinked awards (with awards in File C that are not linked to any award in Files D1 or D2). Each bucket will include one or more files, depending on the size of your download request.

Please note that any files with unlinked awards will include the same columns as the files with linked awards; however, the columns that involve data from Files D1 and D2 will be blank in the files with unlinked awards.

This release occurred on April 20, 2023, with the addition of the unlinked awards files on May 1, 2023.

Bug Fixes

  • Fixed a bug affecting the display of the following data elements in prime award transaction downloads:
    • indirect_federal_sharing
    • funding_amount
    • total_funding_amount
    • non_federal_funding_amount
  • Fixed a bug in the availability of the Database Download’s Subset Database file.
  • Fixed a bug in the click behavior of the Advanced Search “Categories” tab’s dropdown menu.
  • Fixed bugs concerning accessibility in the Advanced Search’s Table/Time/Map/Categories tabs, the Advanced Search’s “Time Period” and “Location” filters, the Agency Profile pages’ “View Outlays” toggle, and the Award Profile pages’ “Additional Information” section.
  • Fixed a bug in the “Learn More” link in the banner at the bottom of each page promoting USAspending training sessions.

April 4, 2023

Announcements

  • USAspending.gov is excited to announce a customer testimonial campaign that showcases creative and meaningful uses of federal spending data. If you have a story you want to share, please let us know in this survey. Your story could be featured in an upcoming USAspending video!

  • Our sister site, [Fiscal Data]https://fiscaldata.treasury.gov), has just launched a new Currency Exchange Rates Converter tool powered by trusted data from the U.S. Treasury’s Treasury Reporting Rates of Exchange dataset. This tool helps easily calculate conversion rates between foreign currencies and U.S. Dollars to give users the most accurate information available. Check it out today!

Website Updates

  • Updated the downloadable list of Congressional Justifications (https://www.usaspending.gov/agency) based on authoritative information from the Office of Management and Budget (OMB).

  • Award Search improvements:

    • Updated the Award ID filter so that searches produce the same results regardless of whether upper-case or lower-case letters are used in the query.
    • Added a tooltip to explain groupings of our filters based on contract award elements, financial assistance award elements, and agency account elements.

Download Updates:

  • Added columns for obligation and outlay amounts funded by the Infrastructure Investment and Jobs Act (IIJA, also known as the Bipartisan Infrastructure Law) to select download packages. The infrastructure data are tagged by Disaster Emergency Fund Code (DEFC) values “Z” and “1”. The data show total amounts funded by the IIJA through the latest agency reporting period. In addition, the analogous and existing columns for COVID-19 obligations and outlays will be renamed to be consistent in phrasing with the new Infrastructure columns.

The new and updated columns will be located and labeled as follows:

  • In Award Summary Files (found in Advanced Search download):

    • obligated_amount_from_iija_supplemental
    • outlayed_amount_from_iija_supplemental
    • obligated_amount_from_covid-19_supplementals (new name)
    • outlayed_amount_from_covid-19_supplementals (new name)

  • In Award Transaction Files (found in Advanced Search download, Award Data Archive download, Award Profile download, and Custom Award Data download):

    • obligated_amount_from_iija_supplemental_for_overall_award
    • outlayed_amount_from_iija_supplemental_for_overall_award
    • obligated_amount_from_covid-19_supplementals_for_overall_award (new name)
    • outlayed_amount_from_covid-19_supplementals_for_overall_award (new name)

  • In Subaward Files (found in Advanced Search download, Award Profile download, and Custom Award Data download):

    • prime_award_obligated_amount_from_iija_supplemental
    • prime_award_outlayed_amount_from_iija_supplemental
    • prime_award_obligated_amount_from_covid-19_supplementals (new name)
    • prime_award_outlayed_amount_from_covid-19_supplementals (new name)

Bug Fixes

  • Fixed a bug in the display of the recipient name in award profile pages. The recipient name is now displayed as the current name (derived from multiple authoritative sources, following the general practice used throughout the website) rather than the name submitted in the transaction data (which can nonetheless be found in download files).
  • Fixed two bugs in the display of the Agency Profiles’ Status of Funds charts. The hover-state drilldown instructions now appear at every level of the drilldown (not just at the first level), and the outlay bar is now clickable across its entire length (rather than just the yellow portion).
  • Fixed a bug in the Award Profiles’ Award Amounts charts. The chart labels are no longer truncated in some cases.

February 1, 2023

Website Updates

  • New page: You can now find all USAspending.gov video tutorials on our new ‘Training Videos’ page! These tutorials will help you understand federal spending concepts and how to use USAspending features. Keep up-to-date with our latest videos here or by subscribing to the USAspending.gov YouTube channel and let us know in the comments what other topics you would like to see covered in future videos.

  • Agency Profile improvements: We’ve updated the interactive Status of Funds chart on our agency profile pages to include information on the individual Treasury Account Symbols agencies and agency sub-components use to classify financial transactions. If you prefer your data in tables, we’ve made improvements allowing you to now drill down into agency data using the table view. Stay tuned for more updates to come!

  • Data Sources download: Have you seen our recently added Data Sources page? For your convenience we’ve now included a downloadable version of this resource. Follow the money and learn all about how federal dollars move from Congress to the American people and ultimately how that data ends up displayed on USAspending.gov.

Bug Fixes

  • Fixed a bug preventing the Custom Account Download (‘Federal Account’ selection option) from correctly collapsing rows down from Treasury Account Symbol to Federal Account. Since multiple rows have the same unique key, the result was rows that appeared duplicative. Users should now be able to assign unique record keys to a given File B row.

Help Shape the Future of USAspending.gov

  • If you are interested in participating in one of our regular user testing sessions, send a message to USAspending.Help@fiscal.treasury.gov. Please include the below in your email:

    • My familiarity with federal spending processes and data is [low, medium, or high].
    • I use USAspending [almost never, sometimes, often].
    • My professional background is […].

January 13, 2023

Website Updates

  • Find answers to your questions about our data sources and more with the new About the Data feature. The modal can be found under “Resources” on the navigation bar and is conveniently accessible throughout the site.

  • Award Search improvements:

    • Based on your feedback, we’ve updated the Award Search user interface to increase the visibility of the various tab views.
    • Added an explanatory tooltip to the Time Period filter on Advanced Search clarifying its functionality, especially for prime award summary results.
    • Added a tooltip to the Location filter explaining the two location field options available: Place of Performance and Recipient Location.
    • Removed “Small Disadvantaged Business” from the Special Designations Recipient Type filter on Advanced Search. The “Small Disadvantaged Business” designation has not been supported since 2008.

  • Download Updates:

    • Added the new field status_of_budgetary_resources_total_cpe to the Treasury Account level file (All-TAS-AccountBalances) of the COVID-19 download allowing users to use the download file to replicate the calculated amounts on the COVID-19 profile page itself.
    • Renamed the adjustments_to_unobligated_balance_brought_forward field in the Custom Account download (File A) to adjustments_to_unobligated_balance_brought_forward_cpe.

Bug Fixes

  • Fixed Award Search bug causing errors in the Categories tab view of Spending by Assistance Listing (CFDA Program).
  • Fixed bug preventing search results from displaying for certain recipients.
  • Fixed Agency Profile bug producing an error message and no data on the Award Spending chart.

API and Technical Notes

  • Created documentation detailing the rules used to assign earliest and latest transactions within a prime award summary. Find the file on this github wiki page in the descriptions of earliest and latest transactions.

November 9, 2022

New Video Tutorials

  • Check out our latest tutorials on our YouTube channel: Did you know that the federal government obligated over $1.1T for grants in FY2022? Learn how to use Advanced Search to filter for grants in our latest YouTube videos! These videos will show you which filters to select, how to find assistance listing (CFDA program) information in the search results, and how to display different breakdowns of the data.

Website Updates

  • New content: “USAspending Data Sources” This new interactive feature lets you follow the money from Congress to the American people to learn all about the data displayed on USAspending.gov.

  • COVID-19 Profile update: In the COVID-19 Profile’s “Total Spending by Budget Category” section, we’ve changed the labels of the last row under each tab from “Unknown Agency (Unlinked Data)” to the following:

    • “Unreported Agencies” when the Agencies tab is selected
    • “Unreported Federal Accounts” when the Federal Accounts tab is selected
    • “Unreported Object Classes” when the Object Classes tab is selected

    These label changes more accurately reflect that the spending shown in this row represents the Budgetary Resources for federal entities that report to GTAS but not to USAspending. The changes have also been added to the COVID-19 Profile Data Sources & Methodology page.

  • New feedback button: We want to hear from you! Use our new feedback button located on the lower right-hand corner of each page to take a brief survey that will help us continue to improve USAspending.gov.

Bug Fixes

  • Fixed a typo that was causing the Award Search downloads to break when filtering by congressional district.
  • Fixed a bug that caused an error to occur when attempting to view subaward spending under the Time tab on Award Search.

API and Technical Notes

  • Updated Federal Account endpoint: Updated /v2/agency/toptier_code/federal_account to add a children list and the following fields:
    • total_budgetary_resources
    • total_obligations
    • total_outlays
    • bureau_name
    • bureau_slug

Help Shape the Future of USAspending.gov

  • If you are interested in participating in one of our regular user testing sessions, send a message to USAspending.Help@fiscal.treasury.gov. Please include the below in your email:

    • My familiarity with federal spending processes and data is [low, medium, or high].
    • I use USAspending [almost never, sometimes, often].
    • My professional background is […].

October 19, 2022

New Content

  • New videos on our YouTube channel: We are excited to share our latest videos on How to Find Government Contracts for Small Businesses. These videos will show you how to use filters in Advanced Search to find federal contracts and spending totals for small businesses. You can organize this information by location, industry, set-aside, and start/end dates to generate insights relevant to your goals.

Bug Fixes and Improvements

  • Upgraded d3, d3-scale, d3-hierarchy, and d3-interpolate to mitigate security vulnerabilities in data visualization charts.

  • Fixed a bug in the new_awards_over_time endpoint that caused the count of new awards to be higher than the actual count.

  • Fixed a bug in the subaward downloads (in Advanced Search, Award Profile Download, and Custom Award Data Download) so that the prime_award_piid and prime_award_fain fields correctly display their code values.

Help Shape the Future of USAspending.gov

  • If you are interested in participating in one of our regular user testing sessions, send a message to USAspending.Help@fiscal.treasury.gov. Please include the below in your email:

    • My familiarity with federal spending processes and data is [low, medium, or high].
    • I use USAspending [almost never, sometimes, often].
    • My professional background is […].

September 2022

New Content

  • See what's new on Fiscal Data: Check out the latest Your Guide to America’s Finances for a breakdown of the trillions of dollars collected and spent by the federal government this year. Learn more about U.S. Revenue, Spending, Deficit, and Debt with interactive explainers that will help answer all your questions about the federal government's finances.

  • Stay tuned for new USAspending.gov video tutorials: In the coming weeks we’ll be adding new tutorials on how to find federal contracts for market research. Watch the latest videos on our YouTube channel.

Website Updates

  • Added an additional SF-133 GTAS data element (SF-133 Line 1020 “Adjustment to unobligated balance brought forward”) to the formula used to calculate the COVID-19 profile’s Total Budgetary Resources and Total Obligations across multiple years to include adjustments to the carry-over amounts. Note that in rare cases where an agency makes a current year adjustment to a prior year ending balance, that adjustment will not be included in this total. See our Data Sources & Methodology page to learn more.

  • We’ve added award outlay amounts (when applicable / reported) for infrastructure-related Loan awards. Find this data on relevant Award Profile pages under the Award Amounts section.

  • It’s easier to find award and subaward data from any state profile page on USAspending. Check out federal spending to your state using our state profile pages and click “View awards to this state” to see all relevant awards on our Award Search page. Use the various award search filters to further refine your results.

  • We’ve updated several Award Search column headers to provide users more clarity:

    • When viewing prime award search results on the Table tab:
      • “Start Date“ has been changed to "Start Date (Period of Performance)”
      • “End Date” has been changed to “End Date (Period of Performance)“
    • When viewing subaward search results on the Table tab:
      • “Sub-Award Date“ is now "Action Date"
    • The Date Range Time Period filter option includes the following changes:
      • “Start Date” is now "Action Date Start"
      • “End Date” is now "Action Date End"

Bug Fixes and Improvements

  • Fixed a bug preventing recipients with ‘null’ DUNS numbers from flowing from the Broker system to USAspending.gov. This fix now groups recipients by Unique Entity Identifier (UEI) rather than by DUNS.

  • Fixed a bug in the Advanced Search filter for CFDA Program (Assistance Listing) so that it produces correct results despite any backend differences in formatting with whitespace.

Help Shape the Future of USAspending.gov

  • If you are interested in participating in one of our regular user testing sessions, send a message to USAspending.Help@fiscal.treasury.gov. Please include the below in your email:

    • My familiarity with federal spending processes and data is [low, medium, or high].
    • I use USAspending [almost never, sometimes, often].
    • My professional background is […].

August 16, 2022

New Content

  • Have you seen our recently launched YouTube channel? Find tutorial videos on how to use USAspending features and learn more about federal spending concepts. Check out the following videos and stay tuned for new content being added soon!

  • We are excited to announce that our sister site Fiscal Data is answering top questions about federal government finances in new ways! In addition to bringing key datasets together on one modern site, Fiscal Data has recently launched the first of four new concepts from Your Guide to America’s Finances that explains the topic of the national debt, with more topics and insights to come!

Website Updates

  • Choose to drill down into agency spending with our interactive Status of Funds chart or get straight to the data in table form with the new chart/table toggle we’ve added to our agency profiles.

  • We’ve made it easier to find to find the awards and subawards for any recipient on USAspending. Just click “View awards to this recipient” from any recipient profile page to see all relevant awards on our Award Search page. Refine your search further with the Time Period and other filters.

  • Based on your feedback we’ve made it clearer that links to profile pages from Award Search do not maintain your selected search filters. Links from Award Search will now open in separate tabs on your browsers to signal that you have left the search page. Please continue sending your feedback and help us improve USAspending!

Bug Fixes

  • Fixed the broken back button on our agency profile pages for mobile devices.

Help Shape the Future of USAspending.gov

  • If you are interested in participating in one of our regular user testing sessions, send a message to USAspending.Help@fiscal.treasury.gov. Please include the below in your email:

    • My familiarity with federal spending processes and data is [low, medium, or high].
    • I use USAspending [almost never, sometimes, often].
    • My professional background is […].

July 2022

Website Updates

  • Updated award and transaction labels - We’ve clarified when an award description refers to a transaction vs. an award summary with the following changes to the website:

    • "Description" headers now read "Base Transaction Description" on the following pages:

      • Award Search - Spending by Prime Award table
      • Federal Account profile pages - Spending by Prime Award table
      • IDV award profile page - Child award orders

    • "Description" header now reads "Transaction Description" on the following prime award transaction level table:

      • Award profile page - Transaction History table

    • "Description" header now reads "Sub-Award Description" for the following sub-award level table:

      • Award profile page - Sub-award table

  • New "Unlinked Award" identifier - Award profile pages for unlinked awards now include tooltips identifying them as such with information on why an award may be unlinked and why federal account information is unavailable for that award.

  • Outlay amounts added to agency profiles - Check out our agency profile pages to see how much federal agencies have outlayed (paid out) from their accounts for a given fiscal year. Click the “View Outlays” toggle on our interactive Status of Funds visualization to see the data and learn more.

  • Award Profile updates - View infrastructure-related spending on relevant award profiles for Grants, Direct Payments, Contracts, Contract IDVs, and “Other” award types.

  • Award Search update - Based on your feedback, we’ve added a horizontal scrollbar to the Award Search results table making it easier to view all displayed columns.

Bug Fixes and Improvements

  • Fixed a bug preventing COVID-19 obligation and outlay data from appearing on the Award Amounts chart displayed on Award Profile pages.

API and Technical Notes

  • Filter on new awards using Award Search - Fixed an issue impacting https://api.usaspending.gov/api/v2/search/spending_by_award_count/ and https://api.usaspending.gov/api/v2/search/spending_over_time/ such that the time period filter now returns results based on the filtered date and filters out awards that are only active over the selected time period. The API now also includes an option to filter on only new awards.

  • Separate award and transaction descriptions – We’ve changed the award_description header in several download files to prime_award_base_transaction_description or transaction_description and updated the Data Dictionary. This update clarifies when an award description refers to a transaction or an award summary.

  • New elements added to downloads - 3 new financial assistance elements have been added to the Assistance_PrimeAwardSummaries and Assistance_PrimeTransactions download files (see our Data Dictionary for detailed information on these elements and more):

    • indirect_cost_federal_share_amount
    • funding_opportunity_number
    • funding_opportunity_goals_text

Help Shape the Future of USAspending.gov

  • If you are interested in participating in one of our regular user testing sessions, send a message to USAspending.Help@fiscal.treasury.gov. Please include the below in your email:

    • My familiarity with federal spending processes and data is [low, medium, or high].
    • I use USAspending [almost never, sometimes, often].
    • My professional background is […].

May 24th, 2022

Website Updates

Bug Fixes and Improvements

  • Advanced Search - Identified issues impacting the NAICS and PSC filters on Advanced Search:
    • Fixed a NAICS bug that prevented users from being able to search on two characters; added a call to clear selected NAICS filters when users reset search.
    • Changed the PSC filter to start searching on one character.

API and Technical Notes

  • Data Dictionary - Added a new Domain Values Code Description column and updated several domain values in the Data Dictionary. We’ve also updated the Data Dictionary download file.
  • Award endpoints - Updated the following endpoints to add non-COVID DEFCs:
    • The account_obligations_by_defc and acount_outlays_by_defc lists in the /v2/awards/{award_id} endpoint now include all DEFCs associated with an award and not just those associated with COVID-19 spending.
    • The child_*_by_defc and grandchild_*_by_defc lists in the /v2/idvs/amounts/{award_id} endpoint now also include all DEFCs.

Help Shape the Future of USAspending.gov

  • If you are interested in participating in one of our regular user testing sessions, send a message to USAspending.Help@fiscal.treasury.gov. Please include the below in your email:
    • My familiarity with federal spending processes and data is [low, medium, or high].
    • I use USAspending [almost never, sometimes, often].
    • My professional background is […].

May 4th, 2022

Website Updates

  • Data Lab sunsetting: Have you heard the news? On September 30, USAspending is realigning with our sister site, Data Lab, to provide an even better user experience! Moving forward, new content will be added on USAspending.
  • Agency Profile enhancements: Hovering over any row on the Status of Funds bar chart will now show the actual Obligation and Budgetary Resource amounts for a given agency sub-component or Federal Account.
  • Custom Account Data download enhancements: We’ve updated the file type names for Files A-C to clarify and improve consistency in how we show our data sources. The file types available for download are now clearly labeled with their business names and corresponding agency submission file names.
  • Send us your training requests: You are now able to request training on how to use USAspending.gov by simply clicking the Sign Up link at the bottom of each page. Make sure to provide the requested background information in the body of your email or we will be unable to process your request.
  • Coming soon: You may have noticed we’ve changed up our homepage. Stay tuned for updates on some exciting new changes and content we’re working on to help answer even more of your federal spending questions.

API and Technical Notes

  • UEI Updates Updated USASpending SQL scripts to account for the renaming of the Broker duns table to sam_recipient
  • Agency Profile Added deprecation warnings to the /api/v2/agency/{toptier_code}/recipients/ endpoint, which will be removed in the future.
  • Documentation Updates Updated documents for the developer bootstrap processes:

March 22nd, 2022

Website Updates

  • Recipient Profile changes: Added a banner to Recipient Profile pages notifying users that they have been updated to use UEI instead of DUNS. Please note that this will change the links to the Recipient Profile pages on the website and you may need to update previously saved links.
  • Update to Agency Profiles Award Spending display: Added introductory text to the Award Spending section on agency profiles to help users better understand the agency award spending data displayed on the table.
  • Outlays added for Financial Assistance awards: Added Outlayed amount (when it is applicable / has been reported) for Financial Assistance awards on the Award Amounts chart displayed on Award Profile pages.

Bug Fixes

  • Duplicate Awards: Fixed a bug that resulted in multiple awards being surfaced for the same Award ID on Advanced Search. This issue impacted awards only – not transactions. Users will no longer see multiple awards for a given Award ID and Advanced Search downloads will contain the same number of rows as the award count shown on Advanced Search.
  • Agency Profile chart update: The blue “Obligations” bar on the Status of Funds chart is no longer visible when Obligations total $0.
  • Keyword Search tooltips: Fixed an issue causing the tooltip on the Loans tab to be hidden behind the table. The tooltip and its content are now visible.

API and Technical Notes

  • Unique Entity Identifier (UEI) updates:

    • Created a new endpoint /recipient/{recipient_id}/ built off of /recipient/duns/{recipient_id}/ and marked the DUNS endpoint as deprecated
    • Updated API contract for /recipient/children/{duns_or_uei}/ and added ability to search on UEI
    • Updated Recipient Lookup ETL to utilize UEI in place of DUNS for grouping recipients
    • Added UEI fields to subaward tables and materialized views
    • Updated Recipient Profile ETL to include UEI logic
    • Updated logic used to determine Recipient Level across the site to use UEI instead of DUNS

  • Sub-award UEI added to Advanced Search: Updated multiple filters used for the spending_by_award, spending_by_award_count and spending_by_category endpoints:

    • Updated Recipient filter on Advanced Search to allow UEI for subawards
    • Updated Keyword filter on Advanced Search to allow UEI for subawards
    • Added UEI fields prime_awardee_uei and subawardee_uei to subaward downloads

March 4th, 2022

Website Updates

  • Infrastructure spending added to Advanced Search: You can now use the Disaster Emergency Fund Code (DEFC) filter to search for infrastructure spending (DEFC “Z” and DEFC “1”).

  • SAM-issued Unique Entity Identifier (UEI) added to site: As the deadline for the federal government’s official transition from DUNS numbers to the new SAM-issued UEI (scheduled for April 4, 2022) nears, we’ve started adding recipient UEIs to the following pages:

    • Advanced Search – Search by recipient UEI using the ‘Keyword’ and ‘Recipient’ filters
      • This currently applies only to prime awards. Sub-award UEIs are scheduled to be added to Advanced Search with our upcoming release.
    • Keyword Search – Search by UEI to find award spending by transaction
    • Recipient Profiles
    • Award Profiles

  • Increased download result count limit: Download limits have been increased from 250K back to the original setting of 500K results.

  • Ascending sort added to Award Search tables: Users can now sort by ascending and descending order on Advanced Search and Keyword Search tables. Tables continue to default to descending order.

  • Outlays added to contract and IDV award profiles: Outlay amounts for non-COVID-19 related contracts and IDVs have been added to the Award Amounts chart on Award Profile pages (when applicable/reported). The chart has also been updated to accommodate four or more layers (e.g., potential award amount, current award amount, obligated amount, and outlays).

Bug Fixes

  • Duplicate Awards Fixed but that was preventing awards from being properly deleted, resulting in duplicate awards surfacing in Advanced Search.
  • COVID-19 Data Sources and Methodology (DS&M) page: Fixed an issue that caused infrastructure DEFC “Z” and “1” to be incorrectly listed as relevant codes for COVID-19 response funding on the COVID-19 DS&M. This issue impacted only the DS&M page – at no time have the values on the COVID-19 Profile page included infrastructure spending. Standardized percentages on Agency Profiles: Updated the percentage figures in the “% of Total” column of the Award Obligations by Type chart to round to one decimal place which is consistent with the percentages shown on the chart labels.
  • Improved tablet and mobile views:
    • Fixed formatting issue that caused the Total Obligations Over Time chart to overlap into the Award Obligations by Type chart. Chart is now correctly centered for mobile and tablet devices.
    • Fixed bug that allowed only a partial view of the UEI banner modal currently displayed across the site. Users are now able to view the full modal

API and Technical Notes

Help Shape the Future of USAspending.gov

  • When preparing to release new features, our team conducts user testing sessions to ensure we’re creating the features you need! If you are interested in participating in one of our regular testing sessions, send a message to USAspending.Help@fiscal.treasury.gov. Please include the below in your email:
    • My familiarity with federal spending processes and data is [low, medium, or high].
    • I use USAspending [almost never, sometimes, often].
    • My professional background is […].

February 8th, 2022

Website Updates

  • New downloadable data elements: Infrastructure Investment and Jobs Act (IIJA) data: Added “Infrastructure Spending” filters to the Custom Account Data download allowing users to filter on DEFC “Z” and “1” to track the infrastructure spending provided in the IIJA.
    • DEFC “Z”: For funding designated for emergency purposes under Section 251 of the Balanced Budget and Emergency Deficit Control Act of 1985 (BBEDCA) contained in Division J – Appropriations.
    • DEFC “1”: For non-emergency funding provided in other sections of the IIJA
  • Unique Entity Identifier (UEI): Added banner alerting users that UEIs – the alphanumeric codes created in the System for Award Management (SAM) that will replace DUNS as the official identifier for doing business with the government - are being added to the site.
    • What changes will you see on USAspending?
      • UEIs have been added to recipient endpoints
      • Download files that currently list DUNS numbers now also include UEIs
      • You will start seeing UEIs alongside DUNS numbers on the following pages:
        • Recipient Profiles
        • Award Profiles
        • Advanced Search
        • Keyword Search
      • You will be able to find federal awards using UEI or DUNS numbers via Keyword Search or by using the 'Keyword' or 'Recipient' filters on Advanced Search
      • URLs to recipient profile pages will become associated with UEIs rather than DUNS — please update any saved links to these pages to avoid service disruption
  • Internet Explorer (IE) no longer supported: Ended support to improve overall site performance. IE users should upgrade to one of the following browsers to continue to access USAspending: Microsoft Edge, Google Chrome, Mozilla Firefox, Apple Safari
  • Award Profile pages:
    • Added outlay amounts to the Award Amounts table for awards where outlay data has been reported. Additional improvements adding outlays to the Award Amounts chart coming later this year.
    • Added links to Agency Profile pages via Contract and Indefinite Delivery Vehicle (IDV) Award Profiles:
      • On IDV Award Profiles – added agency profile links to the “IDV Activity” chart tooltips and the “Orders Made Under this IDV” table for ‘Awarding Agency’.
      • On Contract Award Profiles – added agency profile links for ‘Parent IDV Agency Name’ in the Parent Award Details accordion under the Additional Information section.
  • Agency Profile pages:
    • Added a “Data through” date for each section to clarify which period the obligations displayed come from.
      • Overview and Status of Funds sections show data through the end of the latest closed submission period.
      • Award Spending section shows data through the latest award transaction date for agencies.
    • Added ability for negative obligations to be displayed on the Status of Funds chart.

Bug Fixes

  • Downloads: Fixed a bug that caused download results to sometimes produce result files that did not contain the expected result set. Downloads are now returning full results.

API and Technical Notes

January 18, 2022

Website Updates

  • Updated Agency Profile pages: We’ve updated and expanded agency profiles to provide a more detailed view into federal agency spending.
    • Each agency profile page can now be filtered to view spending by a specific fiscal year (FY).
    • Added three new visualizations in the FY Summary section that show high-level totals for a given agency’s spending.
    • Created a Status of Funds section that shows how an agency’s budgetary resources are distributed among its sub-components (e.g., budgetary resources distributed from the Department of the Treasury to the Bureau of the Fiscal Service) and how much of that money has been spent. Selecting a sub-component updates the chart to display its spending by Federal Account.
    • Created an Award Spending section with tabs for each award type and a nested display to break down spending by Sub-Agency and Office.
  • Ending support for Internet Explorer: Added a warning banner to Internet Explorer (IE) browsers only, alerting users that support for IE will end on Friday, 2/18/2022. IE users should upgrade to one of the following browsers to continue to access USAspending: Microsoft Edge, Google Chrome, Mozilla Firefox, Apple Safari
  • Glossary updates:

Bug Fixes

  • Advanced Search ‘Recipient’ filter: Fixed a bug that would cause the Advanced Search page to return an error when searching by Recipient/DUNS for any award containing an alpha character in the ``ultimate_parent_unique_id` field, which was designated to be a numeric DUNS field.
  • Agency Budgetary Resources Limit Limited the Agency Budgetary Resources endpoint /api/v2/agency/<toptier_code>/budgetary_resources/ to pull information from quarterly periods prior to Fiscal Year 2022.
  • Sub Agency Size Limit Fixed bug that was causing the Agency Sub Agency endpoint to only return 10 results /api/v2/agency/<toptier_code>/sub_agency/

API and Technical Notes

December 7th, 2021

These notes will cover our November 16 and December 7 releases

Website Updates

  • Added a new webpage “Analyzing Equity in Federal COVID-19 Spending: The Opportunity Project” under “Other Resources” on our COVID-19 profile page. The page highlights our collaboration with tech teams, product advisors, user advocates, local government officials and federal data stewards across the country on the mission of analyzing equity in COVID-19 federal funding as part of The Opportunity Project, a public & private sector innovation program led by Census Open Innovation Labs at the U.S. Census Bureau.

Known Issue(s)

  • Download performance:
    • The download result limit has been temporarily changed to 250K while enhancements to our download processes are being implemented.

Bug Fixes

  • COVID-19 Profile:
    • Identified an issue on the COVID-19 profile page that resulted in negative numbers being displayed on the “Unknown Agency (Unlinked Data)” row under the Award Spending by Sub-agency section. The bug has been fixed and the Unknown Agencies row now displays the correct positive numbers for the Award Obligations, Award Outlays, and Number of Awards.
    • Fixed the text alignment for the Total Spending by Budget Category section under the Federal Accounts tab

API and Technical Notes

  • On November 16, Disaster Emergency Funding Code (DEFC) columns were changed to TextField to support more than 2-character DEFCs.

October 27, 2021

Website Updates

Fixes and Improvements

  • Custom Award Data download filters: Fixed an issue in the Custom Award download that caused the Place of Performance filter to be ignored, resulting in large incorrectly filtered downloads.
  • Glossary links in Advanced Search: Fixed a bug that prevented the Advanced Search page from handling glossary links after a search has been performed.

October 5th, 2021

Website Updates

Fixes and Improvements

  • Agency Profile tree view: Fixed a bug where clicking on the tree view displayed under the Object Class section of a given agency profile page returned a large grey box rather than the appropriate drill-down tree view.
  • Agency Profile Landing Page filter: Fixed an issue impacting filter results functionality where results were being filtered on type but not after selecting Enter or the search button. The filter is now consistent with other profile landing pages.

API and Technical Notes

  • Unique Entity Identifier (UEI)
  • COVID-19 Profile Page Updates
    • Added Special Case funds to Disaster Overview endpoint /api/v2/disaster/overview/
    • Added ability to filter Disaster Download to a single DEFC
  • Award Outlays on Award Pages Added award outlay information for overall award, rather than just DEFC specific outlay information:
  • Pipeline Improvements Fixed recurrent error in combine_transaction_search_chunks stage of the pipeline by adding error handling. Attempts to prevent AccessShareLock in Combine Transactions stage of pipeline.
  • Download Improvments Added USSGL fields to File B download. Fields added:
    • obligations_undelivered_orders_unpaid_total
    • obligations_undelivered_orders_unpaid_total_FYB
    • USSGL480100_undelivered_orders_obligations_unpaid
    • USSGL480100_undelivered_orders_obligations_unpaid_FYB
    • USSGL488100_upward_adj_prior_year_undeliv_orders_oblig_unpaid
    • obligations_delivered_orders_unpaid_total
    • obligations_delivered_orders_unpaid_total_FYB
    • USSGL490100_delivered_orders_obligations_unpaid
    • USSGL490100_delivered_orders_obligations_unpaid_FYB
    • USSGL498100_upward_adj_of_prior_year_deliv_orders_oblig_unpaid
    • gross_outlay_amount_fyb
    • gross_outlays_undelivered_orders_prepaid_total
    • gross_outlays_undelivered_orders_prepaid_total_FYB
    • USSGL480200_undelivered_orders_obligations_prepaid_advanced
    • USSGL480200_undelivered_orders_obligations_prepaid_advanced_FYB
    • USSGL488200_upward_adj_prior_year_undeliv_orders_oblig_prepaid
    • gross_outlays_delivered_orders_paid_total
    • gross_outlays_delivered_orders_paid_total_FYB
    • USSGL490200_delivered_orders_obligations_paid
    • USSGL490800_authority_outlayed_not_yet_disbursed
    • USSGL490800_authority_outlayed_not_yet_disbursed_FYB
    • USSGL498200_upward_adj_of_prior_year_deliv_orders_oblig_paid
    • USSGL487100_downward_adj_prior_year_unpaid_undeliv_orders_oblig
    • USSGL497100_downward_adj_prior_year_unpaid_deliv_orders_oblig
    • USSGL487200_downward_adj_prior_year_prepaid_undeliv_order_oblig
    • USSGL497200_downward_adj_of_prior_year_paid_deliv_orders_oblig
    • USSGL483100_undelivered_orders_obligations_transferred_unpaid
    • USSGL493100_delivered_orders_obligations_transferred_unpaid
    • USSGL483200_undeliv_orders_oblig_transferred_prepaid_advanced

September 14th, 2021

Website Updates

  • Mobile optimization: Fixed mobile layout issues on Award Summary pages by stacking Award Description sub-sections responsively for an improved mobile view.
  • General: Added the term “UEI” to the glossary.

Fixes and Improvements

  • COVID-19 profile page : Fixed a bug where users were unable to return to the COVID-19 profile page when using their browser’s back button from various pages.
  • “About Page” scrolling bug: Clicking on a section of the “About Page” sidebar navigation now brings you to the top of your selected section rather than scrolling past the desired content.
  • Award Summary Download: Fixed a recently discovered issue that ensures data included in the Award Summary download file matches the data displayed in the Federal Account Funding table of an Award Summary page.

API and Technical Notes

August 25th, 2021

Bug Fixes

  • Federal Account Profile Page – Fixed a bug where the quarterly balance endpoint was returning the aggregate values of all quarters, rather than the pre-aggregated value of the last quarter.
  • Advanced Search:
    • Sub-Awards View Performance – Improved the performance of Sub-Awards when filtering an Advanced Search by Assistance Listings (formerly “CFDA”) so that results are returned in a timely manner. Previously, Assistance Listing filters with a large number of Sub-Awards were timing out.
    • TAS, NAICS, and PSC filters – Fixed an issue where filters for TAS, NAICS, and PSC were not being retained if users left and then returned to the Advanced Search page from an Award Summary page. Users are now able to change or remove previously selected filters without having to reset their search.
    • ‘Filter Selected’ Message Bug – Fixed a bug where the ‘Filter Selected’ message would appear when a filter was removed, rather than just when a filter was added. Updated the message to state “Filter Updated” when users remove filters.
  • Spending Explorer Link – Removed the broken link from the “Non-Award Spending” table view of the Spending Explorer award breakdown.

API and Technical Notes

August 2nd, 2021

Website

Updates and Fixes

  • Mobile Optimization: This release includes various styling updates to our homepage to help improve the mobile experience, including:
    • Images - Corrected aspect ratio and resolution for homepage images
    • Footer – Condensed the footer to reduce scrolling and resized footer links to make them more accessible.
    • Header – Updated header spacing and alignment on the homepage to align with the rest of the page.

Bug Fixes

  • Spending Explorer: Fixed an issue so that in cases where the "Sum of all Federal Accounts after Top 500" is displayed, the calculation is aggregating the value correctly and only when there are more than 500 accounts.
  • Award Summary tooltips: The COVID-19 tooltip on Award Summary pages now displays Disaster Emergency Fund Codes (DEFC’s) in alphabetical order (e.g., L, N, P, U).
  • Advanced Search filters:
    • Fixed a bug where “No results” was incorrectly displaying when the North American Industrial Classification System (NAICS) checkbox was collapsed.
    • Fixed a bug where Treasury Account Symbol (TAS) codes were not being submitted after clicking Enter.
  • Glossary: Fixed an issue where glossary terms with only Plain Language definitions (but no Official definition) were not displaying if accessed via the “See Also” link under a term with both Plain Language and Official definitions. Terms with only one definition type (Plain Language or Official) will now default to display the definition that is available.

API and Technical Notes

July 13th, 2021

Website

Updates and Fixes

Bug Fixes

  • Federal Account Profile Page: Fixed the Award ID link under the “Spending by Prime Award” section of the Federal Account page. The Award ID link will now navigate users to the Award Profile page.
  • Spending Explorer Tooltip: Removed the following disclaimer from the “Non-Award Spending” tooltip on the Spending Explorer tree view: "The amount shown above is a portion of a larger award that is $0.00." The disclaimer was unnecessary since the tooltip displays a grouping of awards.
  • Spending Explorer Link: Removed the link from the “Non-Award Spending” tree view of the Spending Explorer award breakdown. Users will no longer be sent to a broken page.
  • API Tutorial Link: The API Tutorial link is no longer broken when using a mobile device or smaller browser size.

API and Technical Notes

  • Improved Download Performance: Increased performance of the Custom Award Download and Files A and B on the COVID-19 profile page by reducing the number of static subqueries that are performed for every row. This resulted in overall lower runtimes. Testing results showed the downloads completed in under 4 hours.
  • Agency Office Data: Created a new Django command and ETL job to pull in agency office reference data from Broker to USAspending. Agency office data has also been added to the Elasticsearch Transaction Index to support upcoming website enhancements.
  • Optimized Matviews: Improved performance of the ‘mv_agency_autocomplete’ MatView. Refresh times had previously jumped from 4+ hours per pipeline run to nearly 7 hours.
  • CFDA Filtering in Advanced Search: Fixed a bug on Advanced Search where the search and download are being filtered by FFATA Subaward Reporting System (FSRS) CFDA numbers, rather than the CFDA number(s) of the parent award. Now, when you search a parent award you will be able to view any sub-awards associated with a given CFDA.
  • Format Change to Budget Object Class (BOC) Codes: DAIMS 2.1 requires agencies to stop sending 4-digit BOC codes with the prefix “1” or “2” in front, which used to indicate “Direct” or “Reimbursable” funding. As of June, the new BOC code format is a 3-digit code that allows a trailing “0” at the end. We’ve updated the USAspending BOC loader process to accommodate this change.

June 22nd, 2021

Website

Updates and Fixes

Updates to COVID-19 content and displays:

Search for awards with multiple CFDAs: Since DAIMS 1.3.1, agencies have been able to submit and track multiple CFDA programs for a single modification number and with OMB M-21-03 agencies will be required to submit multiple CFDA for all new awards. Multiple CFDAs were previously added to the Award Summary Pages and recently to the Advanced Search downloads.

  • On Advanced Search, we added the ability to search for awards via any of the CFDA under an award, not just the CFDA associated with the most recent transaction.

Fixes:

  • Fixed images on homepage that were not displaying on a Safari browser
  • Fixed a temporary issue where the download button was disabled on Advanced Search due to styling updates.
  • Fixed a temporary issue on the Spending Explorer where the agency path was displaying an error due to recent changes in the list of agencies displayed.
  • Fixed a bug on the Federal Account profile where caches values were being displayed on initial view, and then only updated to the latest period after refresh.
  • Fixed a bug on the Agency Profile landing page and individual pages where the percent of total government budget was being calculated using a previous period in a quarter instead of the most recent closed period.

API and Technical Notes

  • Custom Account Download: Implemented performance improvements to the Custom Account Download which was failing after 4 hours+ after the addition of the most recent period’s data to the database. Testing results showed the downloads completed in under 4 hours when ‘All Agencies,’ ‘Federal Account’ and ‘All Files’ were selected, and in less time when parameters were limited further.
  • Multiple CFDAs: A new filter was added to the /api/v2/search/spending_by_award/ endpoint. In this new filter, all program_numbers is meant to replace program_numbers. The new filter queries all CFDAs associated with an Award, rather than only the CFDA associated with its latest transaction.
  • Created /api/v2/agency/{toptier_code}/recipients/ endpoint to support Agency v2 Page
  • Created table of pre-aggregated recipient data to support new agency recipient endpoint
  • Fixed bug causing outdated data appearing in downloads by using dates that are set by the Nightly Pipeline to determine "cache" of Downloads
  • Refactored data loaders to change tas_id field name to account_num

June 2nd, 2021

Website

Updates and Fixes

  • New Social Sharing Images: Added new social media preview images for Facebook, Reddit, LinkedIn, and Twitter. Now users will see updated images display when they share pages.
  • Updates to Advanced Search Results & Downloads with DEFC Filter: Updated Advanced Search results and downloads to return accurate outlay and obligation data when a DEFC filter is applied. For example, if a user filters only on DEFC V, they will only see award amounts attributed to that DEFC. Previously, if a user selected DEFC V, they would see any awards with the selected DEFC in the results and downloads.
  • Updates to how new data is revealed on the website at the end of a submission period: Made updates to the “reveal date” so that at the end of a submission period when there is a higher volume of data to load onto the website, pages will not update until all of the data is finished loading to avoid displaying incomplete information. A few ways users may experience this change:
    • On Spending Explorer users will be able to select the latest period from the period picker, typically in the morning after a submission window closes, since it is the first data to complete loading.
    • On the rest of the site, complete data from the new submission period will not update until all of the data is finished loading, typically a day after the submission window closes.
    • Previously, users who went to the site while a large volume of data was still loading may have seen incomplete data on the Spending Explorer, which appeared as a large amount of “unreported data.”
  • Implemented a fix to the display of headers on mobile so that the fiscal year filter and share buttons are accessible on all profile pages and removed the download button
  • The footer link for Release Notes now goes to our wiki for improved access.

API and Technical Notes

May 18, 2021

Website

Updates and Fixes Do you use maps on USAspending? We want to hear from you! The USAspending team is working on enhancements to the website and we would love your feedback. Send an email to usaspending.help@fiscal.treasury.govand tell us more about how you use the map features on the website and what you would like to see in the future.

If you are interested in being a volunteer tester, please include that in your message.

Please include in the body of your email: * My familiarity with federal spending processes and data is [low, medium, or high]. * I use USAspending [almost never, sometimes, often]. * My professional background is […].

API and Technical Notes

May 4, 2021

Website

Updates and Fixes

  • New in Search: American Rescue Plan data Spending data associated with the American Rescue Plan is starting to be reported. You can now search for awards that were reported in March (Period 6) from the Advanced Search page and under Custom Account Download. Use the Disaster Emergency Fund Code (DEFC) Filters and scroll down to check the “DEFC V” option. Explore now.

  • New Page Launched: The Agency Submission Statistics page can now be found under “Resources” in the main navigation. This new feature displays summary information by Fiscal Year, By Period, and By Agency about the data federal agencies submit monthly or quarterly under the 2014 DATA Act.

    • Users can explore:
      • What portion of all federal budgetary resources an agency has in a given FY or period to understand their relative spending
      • When an agency last submitted data for additional context on how recent their data on the website may be
      • The difference between an agency’s reported account balance data on USAspending and what is reported in the Governmentwide Treasury Account Symbol Adjusted Trial Balance System (GTAS)
      • The difference between an agency’s reported award data from their financial system and what is reported in financial assistance and procurement systems Visit the [Data Sources & Methodology section](file:///Users/emilybrents/Downloads/Visit the Data Sources & Methodology section) for a full explanation of the how the summary statistics are calculated and how the table can be used to understand USAspending data.

  • New Content: “What COVID-19 spending does USAspending track?” The Data Sources and Methods section of the COVID-19 Profile page now includes this new section that explains the concept of "covered funds" in the CARES Act to make it clear which supplemental appropriation spending related to the coronavirus response Congress intended to be tracked, audited, and published for transparency and accountability purposes and provides some examples of what you can expect to find on USAspending.

  • Download Update: Multiple CFDAs The award and subaward level downloads are updated on Advanced Search, Custom downloads, and on the COVID-19 Profile page to include all CFDAs associated with a given award instead of just the “primary” CFDA. In the download file, you will now find a single column titled, “cfda_numbers_and_titles” with CFDAs listed in numerical order from smallest to largest, followed by their title. This is consistent with how we display multiple CFDAs associated with an award on each award summary page (Example Award). Also see related API updates below.

  • Spending Explorer: Updated new data reveal Implemented a change so that account data displayed on the Spending Explorer will only update when the nightly data pipeline step to load new submissions is complete, to avoid showing incomplete or “unreported” data while it is still loading. This most impacts users on the day after a submission window closes to ensure the period picker is only selectable for a new period when the data is finished loading.

  • New Data Disclosure: Added a statement below the map on Advanced Search that reads: “NOTE: Data reported by the Department of Health and Human Services (HHS) related to Medicare payments does not reflect the place where "the majority of the work" occurs, as required by USAspending's data model specifications.”

API and Technical Notes

April 21, 2021

Website

Updates and Fixes

  • Sitewide
    • Added DEFC “V” to the website to reflect funding and spending from the American Rescue Plan Act of 2021 (ARPA) as it is reported by agencies. The DEFC V filter is available on Advanced Search (Note users need to scroll to the bottom of the list) and Custom Account Download. In addition, any new reporting for DEFC V in Period 6 will be incorporated into the totals displayed on the COVID-19 Profile page.
    • Updated the filter selection functionality on the Custom Account Download page to make it more intuitive for users applying the DEFC Filter to understand that Account Balance data does not include DEFC. Now, if a user selects only ‘Account Balances’ under file type, they will not be able to select a DEFC. We also added a note to specify if Account Balances is selected, the DEFC column will not appear in the download.
    • Implemented a mobile design enhancement to the COVID Profile Page
    • Fixed a bug on the Agency Profile pages where the Object Class percentage value was not displaying in the tool tip on the tree map visualization.

API and Technical Notes

  • Created Obligations by Award Type API contract
  • Modified Agency Budgetary Resources endpoint to support new bar chart visualization
  • Optimized /api/v2/recipient/duns/ endpoint DB queries to improve performance
  • Reworked calculate_last_completed_fiscal_quarter function across the site to more accurately calculate closed fiscal periods
  • Modified Spending By Geography endpoint to return HTTP 500 error if there is no value for name
  • Fixed bug where downloads from Advanced Search page results lead to empty downloads if search text lead with whitespace characters
  • Updated /api/v2/federal_accounts/ endpoint to include agency detail

April 5, 2021

Website

Updates and Fixes

  • Sitewide
    • Updated the footnote and description of “Unreported Data” on the Spending Explorer.
    • Added a disclosure across website to notify users of the 90-day delay in displaying Department of Defense (DOD) contract award data, subcontract data, and Account Breakdown by Award (File C) data. More information about this delay can be found on the About page Data Quality section. The disclosure was added to the Custom Award Download page, the COVID Profile, the Spending Explorer, and all Federal Account Profile pages with a DOD agency ID (AID). Note that the data on the DOD Agency Profile page is not impacted by this delay, as all the data displayed on that page is sourced from Account Balance (File A) and Account Breakdown by Program Activity & Object Class (File B) data.
    • Fixed bug with Advanced Search page where the download files were returning empty if the entered filter strings started with whitespace characters.

API and Technical Notes

March 23, 2021

Website

Updates and Fixes

  • Sitewide
    • Identified and addressed an issue with multi-year funding and spending calculations on the COVID-19 Profile page and downloads that led to award outlay figures being slightly inflated. Specifically, the issue dealt with accounting for FY 2021 downward adjustments to award outlays. Agencies that reported such adjustments to USAspending now have those numbers incorporated onto the display figures and included in the relevant downloads. Because agencies are not required to submit the downward adjustment fields to USAspending, some figures may still be slightly inflated (i.e., those for which adjustments were made but data was not reported). This will be the case until agencies provide such data. To see an updated version of how our figures are calculated in these sections, please see our Data Sources and Methodology page.
    • Updated the official definition for Disaster Emergency Fund Code in the glossary to include the codes for the Consolidated Appropriations Act of 2020 (P.L. 116-260).
    • Removed any Financing Accounts data surfaced from SF-133 GTAS data throughout the site to ensure consistency. Financing Accounts data has historically been excluded from USAspending TAS reference data since it does not involve budgetary spending.
  • COVID-19 Profile
    • Fixed a bug that led the two banners at the top of the profile to overlap upon scrolling back up the page.
  • Download Center

API and Technical Notes

  • Updated calculations for award outlay amounts across the site
  • Updated /api/v2/disaster/overview endpoint to use new GTAS based calculations
  • Updated filtering on endpoints using the COVID-19 Account Index to ensure awards with negated outlays are filtered out correctly
  • Moved Custom Account download file headers to ensure consistency across files

March 10, 2021

Website

Updates and Fixes

  • Sitewide
    • As of March 3, 2020 data reported under COVID-19 DEFC “U” designated for the Consolidated Appropriations Act, 2021 (Emergency Public Law 116-260) is displaying on the website. Any DEFC U spending that has been reported is incorporated into the COVID-19 Profile page and a DEFC “U” filter was added to the Advanced Search page and Custom Account Download.
    • Fixed a bug that led navigation menus on the left side of the screen (example here) to flicker between different sizes and cover the top header.
  • Federal Account Profiles
    • Removed the Account Description field from the profiles given it did not surface information for any account.
    • Fixed a bug that prevented portions of the page from fully loading unless the user refreshed the page.
    • Fixed a bug that caused FY 2018 Q3 data to be missing from certain Federal Account profiles’ Spending Over Time charts.
  • Advanced Search
    • Fixed a bug that prompted a page error when a user closed the date picker without selecting an end date.
  • State Profiles
  • API Endpoints
    • Fixed a bug that led the description column to extend beyond the designated content area on the page.

API and Technical Notes

February 22, 2021

Website

Updates and Fixes

  • COVID-19 Related Changes:
    • Added DEFC U to the site in anticipation of agency data submissions for COVID-19 spending funded by the Consolidated Appropriations Act, 2021. This data is expected to appear on USAspending.gov in early March with the close of the next submission period.
    • Identified and corrected an issue with multi-year funding and spending calculations on the COVID-19 Profile that led to some figures being inflated in the Overview and Total Spending Sections of the page. Specifically, the issue dealt with adjusting total budgetary resources, obligations, and outlays for FY 2021 de-obligation of unpaid obligations, recovery of prior year paid obligations, and anticipated de-obligation of both paid and unpaid obligations. To see an updated version of how our figures are calculated in these sections, please see our Data Sources and Methodology page.
  • Custom Account Download
    • Fixed a bug that resulted in the wrong file name for custom account downloads.

API and Technical Notes

February 8, 2021

Website

Updates and Fixes

  • Advanced Search:
    • Fixed a bug that prevented users from clicking the "Download" button for some searches with fewer than 500,000 results.

API and Technical Notes

  • Upgraded and removed numerous Python packages
  • Replaced the universal_transaction_matview with the transaction_search Django managed table
  • Separate the Delete and Upsert steps of Elasticsearch ETL
    • Allows the option when running the elasticsearch_indexer.py command to only perform deletes. The command currently has the ability to perform deletes, but upserts are also performed. Added a new parameter --deletes-only to support this requirement.

January 12, 2021

Website

Updates and Fixes

  • Advanced Search:
    • Fixed a bug in the filters of the "Spending by Geography" visualization that caused the page to go blank when certain filters were applied.
    • Fixed a bug that prevented the loading indicator from appearing immediately after submitting a search.
  • Profile pages:
    • Updated the Mississippi flag on its state profile page.
  • About:
    • Added disclosures about known delays in Department of Defense contract data and aggregated locations in Department of Health and Human Services Medicare data, as requested by the Government Accountability Office.
  • Legal:
    • Fixed broken links to our Accessibility, Privacy Policy, and Freedom of Information Act pages.

API and Technical Notes

December 28, 2020

Website

Updates and Fixes

  • Advanced Search:
    • Added a download button to the “Spending Over Time” search results. Download options are available for the three selectable time units: Years, Quarters, or Months.
    • Screenshot of USAspending Advanced Search page on the Spending Over Time tab, displaying a bar graph of spending by year
    • Fixed a bug where the width of the advanced search page was cut off on mobile view.
  • About
    • Added a “Careers” section to the About page, which is also accessible via the website footer menu.
    • Screenshot of USAspending About page displaying the Career section, plain text with an email link.

API and Technical Notes

Data and Download

  • Refactored materialized view definitions and Elasticsearch ETL to simplify the data processing and leverage multiprocessing which improved performance for data preparation and reduced the possibilities for introducing bugs to the Elasticsearch data.
  • Expanded GTAS ETL to pull more account balance fields into USAspending's reference data table.

December 14, 2020

Website

Updates and Fixes

Screenshot of USAspending Download center page displaying a rectangular blue dataset metadata download button

Metadata is machine readable information that describes the contents of datasets available to the public using standardized vocabulary in a reusable structured format.

In publishing USAspending metadata, we followed the structure and format prescribed by the Common Core Metadata Schema v1.1, which was implemented as a result of the Foundations for Evidence-Based Policymaking Act ("Evidence Act") of 2019.

The downloadable file follows this federal standard and includes metadata about the following published datasets on USAspending: Award Data Archive, Custom Award Data, Custom Account Data, Agency DATA Act Broker Submission Files (DABS) files, Agency Financial Assistance Broker Submission (FABS) files, database download, and Application Programming Interface (API) codebase.

A JSON file can be viewed with any text editor application. Each dataset includes metadata such as: title, description, keywords, publication frequency, publisher, contact information, access URL, licensing, and distribution/format.

Bug Fixes:

  • Fixed a bug to remove three (3) financial assistance awards from Advanced Search that had been removed by an agency when it submitted its data.
  • Fixed a bug where the Award ID and Transaction Amount columns were not populating on the Keyword Search page.

API and Technical Notes

  • Created new management command update_missing_covid_awards that ensures that records of awards stored in our Elasticsearch cluster are updated correctly when File C data is missing from the most recent period's submission but present in the previous period.
  • Updated the clone_submissions and copy_submission scripts to include the new is_final_balances_for_fy,submission_windown_id, and disaster_emergency_fund_code fields.
  • Created and populated reporting_agency_tas table for the upcoming About the Data page.
    • Created populate_reporting_agency_tas script to populate the table
  • Created and populated reporting_agency_overview table for the upcoming About the Data page.
    • Created populate_reporting_agency_overview script to populate the table

November 30, 2020

Website

Updates and Fixes

  • Covid-19 Spending Profile:
    • Integrated the data in the “Number of Unlinked Awards” row itself (in the “Total Spending by Budget Category” section with the “Award Spending” filter selected) into the table itself. This row was originally split from the table because of performance considerations, but these issues have since been resolved.
  • Advanced Search
    • Fixed a bug that shifted columns in the “Award Spending by Recipient” table.

API and Technical Notes

Data and Download

  • Fixed a bug that duplicated federal account data from the Small Business Administration (SBA) submission for fiscal year 2020 period 07.

November 17, 2020

Website

Updates and Fixes

  • Advanced Search:
    • Fixed a bug that prevented links to COVID-19 spending Advanced Search results (i.e., searches with DEFC filters applied) from loading the table with the relevant results.
    • Fixed a bug that re-executed a previous search when navigating to Advanced Search using the top navigation menu, instead of presenting the default search view. Returning to Advanced Search using the back button still preserves any previous search criteria, as intended.
  • Covid-19 Spending Profile:
    • Based on user feedback, we renamed the “Award Spending by Agency” section “Award Spending by Sub-Agency.” This emphasis on sub-agency highlights the nested sub-agency details in the table, and it helps distinguish this section from the “Agencies” tab in the “Total Spending by Budget Category” section.
  • Homepage Link:
  • About ("Data Sources" section):
    • Added information about the timing of data submissions and uploads to our website.
    • Updated the Information Flow Diagram (Exhibit 1) that illustrates how financial information flows from various databases to our website.
    • Flowchart diagram describing how financial information makes it to the website.

API and Technical Notes

  • Updated the elasticsearch_indexer Django management command to create and incrementally update FABS + File D award data into an Elasticsearch index.
  • Simplified the transaction_delta_view used in the Elasticsearch ETL by moving several derived columns to the underlying matviews and the universal_transaction_matview (which is a table, not a matview).
    • Added etl_update_date column to transaction_delta_view.
    • Renamed the fields award_description to transaction_description and transaction_fiscal_year to fiscal_year in the transaction Elasticsearch index.
  • Fixed a bug that was causing duplicate requests to the API endpoint for disaster award amount on initial page load on the COVID-19 Profile page.

November 02, 2020

Website

Updates and Fixes

  • Advanced Search:
    • Description column on Advanced Search page results table
    • Added a "Description" column to our "Spending by Prime Award" results table. In cases where the description exceeds the cell size, hover over the cell to see the full description.
    • Fixed a bug in the search result URL hash so that it doesn't generate a new hash when the link is pasted in a new tab.
  • COVID-19 Spending profile:
    • Added a search bar to the "Award Spending by Agency" and "Award Spending by CFDA Program" sections.
    • Fixed a bug in the "Award Spending by CFDA Program" section which, in a small number of cases, had caused rows to be duplicated when relevant awards were updated.
  • Fixed a bug in our State and Recipient Profile pages that prevented the new fiscal year from appearing in the dropdown menu.

API and Technical Notes

  • Created a new Django management command to populate the Elasticsearch cluster, called elasticsearch_indexer to eventually replace the old es_rapidloader command.
    • Increases parallelization by increasing multiprocessing with a process pool.
    • Removed unnecessary code.
    • Reorganizes code into smaller files.
    • Moves es_rapidloader command into single file to be removed at a later date.
  • Updated the universal_transaction_matview to be broken down into chunks when refreshing to increase parallelization
    • Adds chunks parameter to the matview_sql_generator management command
    • Creates new management command combine_universal_transaction_matview_chunks
  • Remove references to the references_cfda database table when loading data into any of the Elasticsearch indices to prevent stale data.

Data and Download

  • Updated the list of agencies that are indexed in the agency profiles and which are able to submit data to USAspending via the DATA Act Broker. Agencies will only appear if they have submitted data.
  • Corrected the date field being used to populate theprime_award_disaster_emergency_fund_codes, prime_award_outlayed_amount_funded_by_COVID-19_supplementals, and prime_award_obligated_amount_funded_by_COVID-19_supplementals fields in the subaward download files across the site. The fields will now be populated as originally intended, in cases where 1) the sub-award is associated with a prime award that received COVID-19 supplemental funding; and 2) the subaward action date is after April 1, 2020.

October 19, 2020

Website

Updates and Fixes

  • COVID-19 Profile page:
    • Fixed a bug in the "Overview" section of the page that was causing the tooltips to display inconsistently on hover.
  • Award Summary pages:
  • Recipient Profile pages:
    • Fixed a bug that caused the "Gathering your data…" message to trigger every time a keyboard key was pressed after selecting one of the award-type tabs.

API and Technical Notes

  • Improved API POST documentation by adding ready-to-use example JSON in the markdown file that users can copy-paste into the query to get started.
  • Created new v2 endpoints to replace legacy v1 endpoints
  • Continued a large refactor effort of the Elasticsearch ETL script for execution performance and extendibility.

Data and Download

  • Fixed a bug in Custom Award downloads so that sub-contracts and sub-grants appear when "All" awarding or funding agencies are selected.

October 05, 2020

Website

Updates and Fixes

  • Advanced Search:
    • Added a column to the Award table that identifies the specific Disaster Emergency Fund Code(s) (DEFC) associated with individual awards (if any).
    • Advanced Search Table
    • Improved speed of the TAS and PSC filters and fixed a bug that caused top-tier counts in these filters to display incorrect results after searching and then clearing a term from the filter search box.
  • COVID-19 Spending profile:
    • Added an updated statement from the Treasury Department to the "Known Data Limitations" PDF linked in the banner.
    • Improved the positioning of the chart labels in the "Overview" section bar graph.
    • Updated the map so that it uses a linear instead of a gradient scale in edge cases with very small numbers of values that did not display well with a gradient scale.
    • Fixed a bug in the "Award Spending by Agency" section that sometimes caused the agency count to differ between the table and the boxed summary number.
    • Fixed a bug that caused the side navigator box to sometimes behave erratically when scrolling
  • Award Summary pages:
    • Added a "Unique Award Key" component to the "Additional Details" section at the bottom of the page.
    • This component includes the "Unique Award Key" field that USAspending uses to identify each award (the same key contained in the URL of each award summary page and the assistance_award_unique_key or contract_award_unique_key in downloads), followed by separate entries explaining each component of the unique award key. Components of the unique award key vary by award type and record aggregation.
    • Added information about grant opportunities from Grants.gov to financial assistance Award Summary pages. This information was previously added to the modal pop-ups in the "Award Spending by CFDA Program" section of the COVID-19 Spending profile page.
    • Updated the IDV pages so that if COVID amounts are displayed in one tab of the Award Amounts section then they are displayed in the second tab as well.
    • Fixed a recently introduced bug that had caused the links in the Federal Accounts module to not function.
  • Spending Explorer:
    • Updated the visualization to account for results that exceed the displayable limit of 500 in the Recipients and Awards categories (previously they were omitted, leading to incorrect % calculations and summed totals in cases when more than 500 applied). Now, the sum of all results beyond 500 are captured in a box/row labeled "Sum of all [Awards/Recipients] after Top 500" and percentages and totals are accurate.
    • Spending Explorer
  • Custom Award Data Download:
    • Fixed a recently introduced bug that was leading to unexpected blank download files in the Custom Award Download.
  • Custom Account Data Download:
    • Added a filter for Disaster Emergency Fund Codes (DEFCs), which identify supplemental appropriation bills targeting disasters and emergencies such as COVID-19. Currently only COVID-19 DEFC are filterable, though codes for future disasters will be added at the appropriate time.
    • DEFC Filter in Custom Account Download

API and Technical Notes

  • Improved the performance of endpoints that power the TAS and PSC filter trees in Advanced Search:
  • Promoted better security of API documentation site by updating various javascript libraries and adhering to best practices for secure session cookies.
  • Modified /api/v2/award/<award_id>/ to no longer return disaster_emergency_fund_codes since it partially duplicates other fields in the response.
  • Increased the upper-limit of results returned by the Spending Explorer API /api/v2/spending/ from 500 to 1,000.
  • Updated the DABS submission date windows ETL to also accept a CSV file for loading new dates into the USAspending database.
  • Added a foreign key relationship between submission_attributes and dabs_submission_window_schedule for streamlining SQL queries.
  • Updated the custom account download generation endpoint /api/v2/download/accounts/ to accept Disaster and Emergency Fund Codes (DEFCs).

September 21, 2020

Website

Updates and Fixes

  • Updated our page titles and descriptions to provide more granular information about what each page contains. This affects search engine results for USAspending pages as well as the titles for each page within a web browser.
    • Search Engine Optimizations
  • Added a disclaimer for mobile users so they know that the site is best experienced in landscape view.
  • We fixed several broken agency hyperlinks in the Spending Explorer and COVID-19 Spending page that attempted to take users to nonexistent agency profile pages for the small number of agencies that do not submit data directly to us but whose Treasury Account Symbols (TAS) are nonetheless included in the submissions of larger agencies. Only agencies that submit data directly to us have agency profile pages. The agency names in question are still displayed on-screen, they just no longer contain hyperlinks.
  • COVID-19 Spending profile page:
    • Added a modal pop-up for all rows in the Award Spending by CFDA Program (Assistance Listing) table. The modals show detailed information about the selected CFDA program and provide links to beta.sam.gov and grants.gov related to the CFDA. Users can still click a link at the bottom of the modal to go to Advanced Search and see all COVID-19 awards associated with the selected CFDA program (the previous functionality for the hyperlink prior to the addition of the modal).
      • CFDA Details
    • Added informational tooltips throughout the COVID-19 Spending profile page, found by hovering over elements of the Overview bar chart as well as all 'i' icons.
      • Award Overview
    • Fixed a bug that mistakenly showed the old outlay balance for an award in cases when the award was omitted from later reporting periods in the fiscal year. The outlay balance now correctly displays as zero for such awards, where the award displays at all.
    • Added a single row below each of the following tables to account for the difference in the topline sum and the sum of the table entries themselves. In all cases, adding this additional row to the sum of all table entries will yield the relevant high-level amount displayed in the box above the table.
      • Total Spending by Budget Categories (Total Spending filter selected)
        • Here the difference is caused by the gap between the obligations/outlays in GTAS (used for the topline numbers) and File B (table contents).
      • Total Spending by Budget Categories (Award Spending filter selected)
        • Here the difference is caused by whether File C data is linked to File D1/D2 data. The topline count includes linked + unlinked awards; the table row counts include only linked awards.
      • Award Spending by Recipient (Recipients tab, All Awards filter)
        • Here the difference is caused by the topline numbers including linked and unlinked File C data, while the table rows rely on linked data.
      • Award Spending by Agency (All Awards tab)
        • Here the difference is caused by the topline numbers including linked and unlinked File C data, while the table rows rely on linked data.
      • Award Spending by Agency (specific award type tabs)
        • Here the difference is caused by missing funding agency information on a small subset of linked awards.
      • Award Spending by CFDA Program (All Assistance Awards tab)
        • Here the difference is caused by the topline numbers including linked and unlinked File C data, while the table rows rely on linked data.
  • Spending Explorer:
    • Added a 'Non-Award Spending' category to the Recipient and Award drilldown layers so that the full spending amount contained in the higher level category is represented. Previously, the File B data was discarded when Recipients or Awards were invoked, leading to confusion.
      • Spending Explorer
  • State profile pages:
    • Added information about Face Value of Loans to all state profile pages, as well as an explanatory tooltip. The values are scoped to the selected time period in the sidebar navigator.
  • Advanced Search and Keyword Search:
    • Fixed an edge-case bug in the NAICS filter so that searching for a term does not invalidate previously selected items in the hierarchy that match the search term.
  • Award Summary pages:
    • Updated our logic for usage of the COVID-19 Spending badge. Consistent with our usage across the site, if net obligations and outlays from relevant Disaster Emergency Fund Codes (DEFCs) are $0, then the award is not labeled with the badge (since it is not associated with COVID-19 spending). For contract IDVs, the badge will be displayed if anything associated with the IDV (i.e., any child awards or IDVs, grandchild awards, or the IDV itself) has non-zero obligations or outlays.

API and Technical Notes

Data and Download

  • Updated object_class_code in Account downloads and object_classes_funding_this_award in Award downloads to be consistent with standard government convention for displaying object classes: the codes now contain a decimal place. For example, previously object_class_code might have contained "130"; now, it would contain "13.0" instead.

September 08, 2020

Website

Updates and Fixes

  • Modernized our routing approach so that we no longer rely on hash routing. In effect, this means that all of our URLs have changed from the old 'https://www.usaspending.gov/#/search' format to the new 'https://www.usaspending.gov/search' format. We expect this change to improve the searchability of our site pages on various search engines. Note that previously saved links that contain the '#' will remain functional and re-route to the new hashless URL automatically.
  • Updated all spending maps throughout the site so that a message displays if WebGL is disabled that makes it clear that you need to enable WebGL to properly view the map.
  • Spending Explorer:
    • Updated Spending Explorer so that it now allows users to select monthly data increments starting with April 2020 (Period 07) activity—the first period many agencies began reporting monthly per the requirements in OMB M-20-21. Per the memo, most large agencies will be reporting monthly from now on, while most small agencies are allowed to continue to report quarterly until FY22, at which point all agencies will report monthly. Throughout USAspending (including in Spending Explorer) data for any agencies reporting monthly will be available as soon as that month's reporting period has closed; data for agencies reporting quarterly will be available as soon as the quarter's reporting period has closed. A schedule of reporting periods can be found here.
  • Advanced Search:
    • Updated the DEFC filter so that it only surfaces COVID-19 awards that received net funding contributions from COVID-19 DEFC. Previously, the filter merely indicated an association with a COVID-19 DEFC, even if the dollars from COVID-19 DEFC over multiple transactions netted to $0. Because such awards ultimately received no COVID-19 supplemental funding, they should not surface from this filter. This change aligns the COVID-19 award counts on advanced search and the COVID-19 profile page (which already filtered out awards that netted to $0 from COVID-19 supplemental funding).
    • Fixed a bug that caused the 'Filter Selected' message to behave opposite to expectations.
    • Fixed a bug of recent vintage in the 'Prime Recipient Name' link in the Subaward Table such that the link now resolves properly
  • COVID-19 Spending profile:
    • Added tooltips with additional information to the Overview bar chart.
    • Updated the recipient search bar such that it recognizes special characters like commas and hyphens (previously, special characters were ignored and not used in the search). With this change, the list of recipients between the online table and the download package above the table now match exactly (when the table is presenting less than the on-screen maximum of 10k items).
    • Added a 'Back to the COVID-19 Spending profile' button and a 'Share' button to the Data Sources & Methodology page.
    • Updated the 'Recipient Locations' map to use a quantile scale. Also updated the map legend to round consistently and display as a gradient rather than a series of six buckets.
    • Fixed a bug that caused excessive backend API calls to be made by the frontend.

API and Technical Notes

August 24, 2020

Website

Updates and Fixes

API and Technical Notes

Data and Download

  • Added the following new Glossary entries:
  • ZIPs and file names generated from Custom Account Data downloads now reflect Fiscal Year, Quarter, and Period (rather than just Fiscal Year and Quarter), as appropriate.
  • Updated the following column header names in sub-contract downloads throughout the site for consistency with other headers in the file:
    • national_interest_action_code is now called prime_award_national_interest_action_code
    • national_interest_action is now called prime_award_national_interest_action
    • period_of_performance_potential_end_date is now called prime_award_period_of_performance_potential_end_date
  • Added two new columns in Account Data by Award download files throughout the site that capture the date and fiscal year of the action_date of the most recent FABS or FPDS transaction tied to that award:
    • award_latest_action_date
    • award_latest_action_date_fiscal_year

August 07, 2020

On August 7, 2020 we deployed new features, enhancements, and updates based on user feedback, as well as published COVID-19 spending data in accordance with OMB Memorandum M-20-21: Homepage

Website

Updates and Fixes

  • COVID-19 Spending profile page:
    • Though we have added COVID-19 spending information to several sections of USAspending, we created an entirely new page to serve as a hub and summary of COVID-19 spending. The page can be accessed under a new section in the Profiles dropdown menu.
      • Profiles
    • The page (along with COVID-19 data throughout the site) will be updated at the end of each month as we receive additional data from agencies; to check the latest period of activity covered, see the data through [date] notes throughout the page.
    • Created a bar chart in the Overview section that shows high-level totals of COVID-19 spending.
      • Overview
    • Created a Total Spending by Budget Category section that breaks down spending by Agencies, Federal Accounts, and Object Classes. A secondary filter breaks down these categories into Total Spending, Award Spending, or Loan Spending.
      • Budgetary Categories
    • Created an Award Spending by Recipient section with tabs for Recipient Locations and Recipients, and filters for award type.
      • The Recipient Locations tab contains a color-coded map based on the amount of spending, and it can be filtered by area type (state, congressional district, county), spending type (award obligations, award outlays, face value of loans), and amount type (total spending, per capita spending). This map, and the Advanced Search map, now allow users to zoom out when viewing by congressional district or county in order to view all counties or districts nationwide in a single view.
      • The Recipients tab contains a searchable list of award recipients.
      • Recipient Map
    • Created an Award Spending by Agency section with filters for award type and nested display of sub-agencies.
      • Agency Table
    • Created an Award Spending by CFDA Program (Assistance Listing) section with filters for award type. Each CFDA entry contains a link to Advanced Search results with that program pre-selected.
      • CFDA
    • Assembled a list of Other Resources with government sites related to COVID-19.
    • Created a Data Sources & Methodology (DS&M) page that explains all calculations on the page and the datasets powering it.
    • Alongside the DS&M page, for those who may have questions about the limitations of our COVID-19 data, we provided a PDF with disclosure statements from relevant federal agencies regarding the known limitations of their COVID-19 data submissions.
  • Advanced Search – COVID-19 updates:
    • Added a filter for Disaster Emergency Fund Codes (DEFCs), which surfaces awards that used COVID-19 supplemental appropriations to fund all or part of their activity. Note that many awards that surface based on this filter were funded only in part by these bills (particularly long-standing awards where only recent modifications were funded by the bills). Note also that while COVID-19 supplemental appropriation spending is only available at the award level (not the transaction level), we have chosen to limit prime transactions and subaward transactions surfaced in Advanced Search when the DEFC filter is used to those with Action Date after 4/1/2020, given that our COVID-19 spending data only covers spending activity beginning in April 2020.
      • Advanced Search Filter
    • Added columns for COVID-19 Obligations and COVID-19 Outlays in the search results Award Table. These capture the exact amount of the award funded by the COVID-19 supplemental appropriation bills. Analogous fields are also present in all award downloads.
      • Advanced Search Table
  • COVID-19 Award Summary pages:
    • Created a purple COVID-19 Spending badge that appears when one or more COVID-19 Disaster Emergency Fund Codes (DEFCs) have funded an award. The badge will not appear in cases where COVID-19 funding nets to $0.
    • Updated the Federal Account Funding tab to add outlay and DEFC columns (which are also present in the analogous file in the page download).
    • Added COVID-19 Obligated Amount and COVID-19 Outlayed Amount as elements to the Award Amounts chart.
      • Award Summary
  • Other COVID-19 related changes
    • Updated the USAspending.gov homepage to feature COVID-19 spending content and explanations of all the COVID-19 spending related changes on the site.
    • Updated the Data Dictionary; it now includes each of the numerous changes and additions made to our data downloads related to capturing COVID-19 spending.
    • Added monthly files to the Agency Submission Files section of the Download Center; per OMB M-20-21, most federal agencies are now submitting DATA Act data on a monthly basis, starting with FY20 P07 data. Monthly files are listed with a period designator corresponding to the month in the federal fiscal year whose activity and balances they capture (e.g., P01 = activity that occurred in October, P02 = activity that occurred in November … P12 = activity that occurred in September). Some smaller agencies will continue to submit data on a quarterly basis until the end of FY21, after which all agencies will submit monthly.
      • Raw Files

API and Technical Notes

Data and Download

  • COVID-19 Spending profile page:
  • New award summary/award transaction/subaward elements added to the noted download type
    • Prime Award Summary Level Downloads
      • disaster_emergency_fund_codes
        • captures any DEFC that have provided (non-zero) funding for the award
      • outlayed_amount_funded_by_COVID-19_supplementals
        • captures any outlays under the award funded by COVID-19 supplemental appropriations
      • obligated_amount_funded_by_COVID-19_supplementals
        • captures any obligations under the award funded by COVID-19 supplemental appropriations
      • award_latest_action_date
        • captures the latest action date out of all transactions under the award
      • award_latest_action_date_fiscal_year
        • captures the fiscal year of the latest action date out of all transactions under the award
    • Prime Award Transaction Level Downloads
      • disaster_emergency_fund_codes_for_overall_award
        • captures any DEFC that have provided (non-zero) funding for the overall award (not necessarily this specific transaction)
      • outlayed_amount_funded_by_COVID-19_supplementals_overall_award
        • captures any obligations under the overall award (not necessarily this specific transaction) funded by COVID-19 supplemental appropriations
      • obligated_amount_funded_by_COVID-19_supplementals_overall_award
        • captures any outlays under the overall award (not necessarily this specific transaction) funded by COVID-19 supplemental appropriations
    • Prime Award Transaction and Summary Level Downloads
      • object_classes_funding_this_award
        • captures any object classes funding the overall award (not necessarily a specific transaction)
      • program_activities_funding_this_award
        • captures any program activities funding the overall award (not necessarily a specific transaction)
      • recipient_county_name (contract downloads)
        • captures the county name of the recipient's business address
    • Subaward Downloads
      • prime_award_disaster_emergency_fund_codes
        • captures any DEFC that have provided (non-zero) funding for the prime award
      • prime_award_outlayed_amount_funded_by_COVID-19_supplementals
        • captures the total COVID obligations (meaning DEFC = L M N O P) associated with the prime award across all time
      • prime_award_obligated_amount_funded_by_COVID-19_supplementals
        • captures the total COVID outlays (meaning DEFC = L M N O P) associated with the prime award across all time
      • prime_award_latest_action_date
        • captures the latest action date of the associated prime award
      • prime_award_latest_action_date_fiscal_year
        • captures the FY of the latest action date of the associated prime award
      • prime_award_object_classes_funding_this_award
        • captures any object classes funding the overall prime award (not necessarily a specific sub-award)
      • prime_award_program_activities_funding_this_award
        • captures any program activities funding the overall prime award (not necessarily a specific sub-award)
      • prime_awardee_county_name
        • captures the county name of the prime recipient's business address
  • Custom Account Download now only surfaces balance data from the requested period/quarter. Previously, it also surfaced certain balances from previous periods/quarters in the same fiscal year: namely, when an agency had reported a TAS (Account Balances file), combination of TAS + Object Class + Program Activity + DEFC (Account Breakdown by Program Activity & Object Class file), or combination of TAS + Object Class + Program Activity + DEFC + Award (Account Breakdown by Award file) in a previous period/quarter but not in the selected period/quarter.
    • Due to this change, we have replaced last_reported_submission_period with submission_period in Account Balances and Account Breakdown by Program Activity & Object Class download files.

July 13, 2020

Website

Updates and Fixes

  • Fixed some bugs in the Award Summary page Contract and Grant Activity Charts so that the 'Action Type' field displays as intended and the graph displays certain edge cases that were previously not appearing properly.

API and Technical Notes

Data and Download

  • Updated GTAS loader with many more GTAS datapoints
  • Created New DABS date window reference data table: dabs_submission_window_schedule

June 29, 2020

Website

Updates and Fixes

  • Updated the Recipient Profile pages to add the amount of loan dollars (known as the loan face value) that a given recipient received in the selected time period as well as the count of loan transactions. Note that loan face value is a distinct concept from the loan component of the ‘Total Awarded Amount’ above it (see the tooltips on the page for an explanation of the difference and interrelationship).
    • Recipient Profile
  • Fixed a recently-introduced bug that caused filter titles and checkboxes to disappear from the Federal Account Profile pages.
  • Added tooltips to the TAS, NAICS, and PSC filters on Advanced Search.

API and Technical Notes

  • Created new DEFC reference data endpoint describing all Disaster and Emergency Funding (DEF) Codes.
  • Added new DEFC columns to the Award response.
  • Added total_face_value_loan_amount and total_face_value_loan_transactions to the recipient endpoint.
  • recipient_lookup SQL scripts now limit data to no older than FY2008 (2007-10-01 and onward). This is not intended to alter any existing functionality, but will prevent very old data from being considered for recipient "alternate name" logic.

Data and Download

  • Fixed a bug introduced in April that caused certain FABS record deletions to remain unprocessed in USAspending. Such delete requests are now being properly processed and the previously-missed delete requests are being re-run to close the delete gap.

June 15, 2020

Website

Updates and Fixes

  • Added robust burn-up charts in our contract and grant award summary pages that show the running total of spending over time for the award. Capabilities include:
    • Color-coded date lines for the Start of the award; Current End of the award; Potential End of the award (contracts only); and Today's date.
    • Modal pop-ups for blue dots representing individual award transactions. Each pop-up lists the transaction's Modification Number; Action Date; Obligated Amount; Total Obligations to Date; Action Type; and Description. Blue arrows at the bottom of the modal move you through multiple transactions on that day, if applicable.
    • Modal dots are tab-navigable for accessibility and as an alternative for busy charts with numerous dots in close proximity. Try it out by clicking on any individual dot and pressing tab (or shift+tab) to move to an adjacent dot.
    • 'View transactions table' button scrolls to the source data for the chart in the 'Award History' section.
    • View this update on a grant summary page from the Department of Commerce funding the Joint Institute for the Study of the Atmosphere and Ocean at the University of Washington.
      • Activity Chart
  • On the Advanced Search and Federal Account award tables, updated the loan tab columns to use more accurate language for the loan context, and added a useful explanatory tooltip.
  • On Contract and Contract IDV Award Summary Pages, added the National Interest Action field to the Additional Details section. This field was already present in the data and in the download; the only change here was adding it on the page.

API and Technical Notes

  • Added national_interest_action and national_interest_action_description to the awards endpoint.
  • Updated the award transaction loaders to improve accuracy.
  • Added a new table, disaster_emergency_fund_code, to record all valid Disaster Emergency Fund codes. This update does not impact the API currently, but will be used to support future features that track COVID spending.
  • Updated our elastic search freetext fields (including recipient, award ID & keyword filters on Advanced Search as well as the Keyword Search page) to sanitize additional characters.
    • Characters now sanitized (meaning they are ignored from inputs and not part of an executed freetext search): &|{}()*?:\"+[]<>
    • We initially also sanitized these characters, but after user feedback decided to treat them using escape characters instead so they can still be used in freetext searches (e.g., to surface award IDs that contain - or ~): !-~/^
    • If any of the removed characters are useful to you in surfacing specific awards or recipients, feel free to write us and we will consider adjusting the sanitized characters further.

Data and Download

June 01, 2020

Website

Updates and Fixes

  • We launched a major revamp to the North American Industry Classification System (NAICS), Product or Service Code (PSC), and Treasury Account Symbol (TAS) filters in Advanced Search. This upgrade was in response to longstanding user requests related to discoverability and taking advantage of the embedded NAICS and PSC hierarchies. New features include:

    • Filters contain a complete list of NAICS, PSC, or TAS (respectively) tied to federal awards in our dataset.
    • Filters incorporate nested hierarchies, so parent categories of NAICS, PSC, or TAS are easily selectable instead of limiting searches to individual codes. Click a checkbox on any higher-level category to select everything below it.
    • You can easily search across all NAICS, PSC, or TAS codes and titles to explore the universe of codes before you lock in your filters.
    • Here's the new NAICS filter:
      • NAICS Filter

  • Added a social share button to most USAspending pages with permalinks. The share button allows you to easily copy, e-mail, or share the current URL on social media.

  • Added a hyperlink to the relevant Agency Profile page for each search result row in the Advanced Search Award Table.

    • Agency Links

API and Technical Notes

May 18, 2020

Website

Updates and Fixes

  • Integrated Elasticsearch into all parts of Advanced Search (except for Sub-Awards, which will remain on Postgres for the time being). This is the culmination of a substantial technical effort and will directly pave the way to future improvements to Advanced Search coming later this year. In the meantime, users will notice improvements for most if not all aspects of search performance: quicker results on screen for the initial search, as you move between the tabs or categories after executing a search, or as you change search filters and re-execute a search. Speed improvements are especially noticeable in the 'Categories' tab and the 'Map' tab (especially the County and Congressional District levels) and with any of the formerly slow filters. Even if you are not a habitual user of Advanced Search, we encourage you to explore it once again now that we’ve built these improvements under the hood!

API and Technical Notes

Data and Download Notes

  • Added the columns national_interest_action_code and national_interest_action to Account Breakdown by Award and Sub-Contract downloads throughout the site. These elements were already captured in Prime Contract data downloads throughout the site, so this is simply a matter of carrying them forward to these additional downloads (which also carry Prime Contract information). These two elements map to 'National Interest Action' in FPDS, an element that captures whether a contract is related to particular 'national interest' items—often natural disasters.

May 04, 2020

Website

Updates and Fixes

  • Advanced Search:
    • Added a toggle to the Map tab to switch between "Total Spending" and "Per Capita Spending" so that users can account for population when comparing spending amounts across geographical regions. This affects both the coloring of the map as well as the individual numbers returned upon hovering over specific regions. This toggle functions with each location level (State, County, and Congressional District) and location category (Place of Performance and Recipient Location).
    • Map
  • Added a chart in the "CFDA Program" section of financial assistance Award Summary pages that have multiple CFDA programs (example award). This chart appears after you select a specific CFDA program; the chart displays the proportion of funding that the selected program provides out of total funding from all CFDA programs for that award.
    • CFDA
  • Added a subheading to certain Recipient Profile pages whose DUNS number is associated with multiple names in the historical award transaction data. Clicking the subheading link opens up a modal that lists all these names in alphabetical order. Here's a page that features the new subheading.
    • Recipient Aliases
    • Recipient Name List
  • Fixed an error of recent vintage on the Federal Account Profile pages where the page would break when a program activity or object class filter was engaged.

API

Updates and Fixes

  • Added /api/v2/references/award_types/ endpoint based on user requests. The new endpoint lists all categories of award types and all members of each category. This is useful in combination with any other endpoint that leverages the award_types parameter. The API contract for the new filter is located here.
  • /api/v2/references/filter_tree/tas and all "filter_tree" endpoints now support a filter query parameter. When present, only results with the filter string in the id or description will be returned.
  • Added alternate_names to the Recipient Endpoint.
  • Keyword Search and Spending by Geography are now supported with Elasticsearch.
  • Fixed a bug with character sanitization in Keyword Search that caused it to fail when certain special characters were used.
  • Updated the fields indexed by Keyword Search. You can see the fields used here.

Data and Download Notes

  • Updated all account downloads in the following ways:
    • Added a new column called owning_agency_name that makes it clear which agency we bucket each TAS/Federal Account (and its budgetary resources or obligated dollars) with throughout the website (e.g., Agency Profile page, Spending Explorer).
    • Changed agency_identifier to agency_identifier_code for consistency with how other codes are labeled.
    • Changed agency_name column to agency_identifier_name for clarity, since this is the name associated with the agency_identifier_code TAS component.
    • Changed allocation_transfer_agency_identifier to allocation_transfer_agency_identifier_code for consistency with how other codes are labeled.
    • Changed allocation_transfer_agency_name to allocation_transfer_agency_identifier_name for clarity, since this is the name associated with the allocation_transfer_agency_identifier_code TAS component.
  • Updated how TAS are collapsed into Federal Accounts (e.g., in Custom Account Download) in edge-case situations where the TAS being collapsed have more than one reporting_agency_name, budget_function or budget_subfunction. Previously, this prevented full collapsing, and more than one row would remain (one for each separate combination of reporting_agency_name, budget_function or budget_subfunction found among the TAS being collapsed); now, rows are fully collapsed as expected, and the different values found among the TAS are included in the single relevant cell at the Federal Account level, separated by semi-colons.
  • Slightly tweaked our method of determining agency ownership of Federal Accounts that have shared AID (e.g., 011). This update only affected two edge cases: after the change, Federal Account 011-0035 is linked to Health and Human Services and Federal Account 011-1475 to Treasury.
  • Added two new columns to award-summary-level award downloads and account breakdown by award account downloads to better provide time context for the awards. This is particularly useful for financial assistance awards that may lack period of performance information (which is currently optional for financial assistance):
    • award_base_action_date
    • award_base_action_date_fiscal_year
  • Added three new columns to all sub-award downloads to better provide date context for the associated prime award:
    • prime_award_period_of_performance_start_date (added to sub-contract and sub-grant downloads)
    • prime_award_period_of_performance_current_end_date (added to sub-contract and sub-grant downloads)
    • period_of_performance_potential_end_date (added to sub-contract downloads only)

April 20, 2020

Website

Updates and Fixes

API

Updates and Fixes

Data and Download Notes

  • Improved our treatment of TAS and Federal Accounts by ensuring that each is assigned a single owning agency and propagating this assignment throughout the site. The previous treatment led to several oddities and was inconsistent between sections of the site to boot (which led to small differences in $ counts by agency between any sections that varied in assignment logic). The assignments are now handled just once during the data load and thus propagate consistently across the site. In the next sprint release, we will add an 'owning_agency_name' to the account download to make the agency TAS and Federal Accounts assignments crystal clear. Here are specific pages that were affected by the assignment logic update:
    • Custom Account Data Download:
      • Previously, not every TAS or Federal Account was assigned an agency owner. Now they are, and all TAS within a Federal Account have the same agency owner as the Federal Account as a whole.
      • Previously, the Federal Account dropdown for agencies that shared an AID with other agencies (e.g., Peace Corps) showed no results; now, the expected results are shown.
      • Previously, a Federal Account could be listed twice in downloads if the TAS underneath it were assigned to multiple agencies (the $ amounts were also split according to these assignments, so $ data was not duplicated); now, this cannot happen, since all TAS under a Federal Account have the same assignment as the Federal Account.
  • Spending Explorer:
    • The previous treatment left some data unassigned to any owning agency, which led to an overlarge 'Unreported Data' section when choosing the Agency starting point and to missing unassigned accounts when drilling down to affected agencies. Now, agencies contain all expected accounts and the 'Unreported Data' $ amount is the same for each of the Budget Function, Agency, and Object Class starting points.
  • Federal Account Profile landing page:
    • The 'Managing Agency' column now shows the correct assignment for each Federal Account per the new logic. This particularly improves agency assignments for Federal Accounts whose AID are shared by more than one agency (e.g., 011, 016, 033).
  • Updated the Federal Account landing page to display all federal accounts that have been submitted since DATA Act data collection began in FY17 Q2. Previously, only accounts submitted during the current fiscal year were shown on the landing page (though the other account pages were still accessible via URL or when linked directly from other sections of the site), which limited the discoverability of older accounts.
  • Updated the Custom Account Data Download federal account dropdown in a similar manner: it now displays all federal accounts that have been submitted to us across all years, so that older accounts are directly selectable.
  • Fixed a bug that affected the (recently-added) Place of Performance option on Custom Award Data Download. The bug produced a blank contract download file when the 'All Foreign Countries' option was selected, instead of the expected data. With the fix, the complete and expected results are returned.
  • Fixed a bug that prevented certain address fields from displaying at the top of award summary pages for foreign recipient addresses.

April 06, 2020

Website

Updates and Fixes

  • Advanced Search Agency Filter
    • Fixed a bug that occurred when searching for a Sub-Agency whose name is shared by more than one Top-Tier Agency (e.g., many agencies include an Office of the Inspector General Sub-Agency). Instead of showing results for the specific Top-Tier Agency + Sub-Agency combination you had selected, all records matching the Sub-Agency name would be shown across the government. The filter now applies the full Top-Tier + Sub-Agency combination selected.
    • To make the selected Top-Tier + Sub-Agency combination clearer, we updated the Sub-Agency selected filter cards to include the associated Top-Tier Agency; previously, they just showed the Sub-Agency Name.
    • New Awarding Agency Filter Text
    • View this update by visiting our Advanced Search page.
  • To reduce confusion and redundancy, updated Advanced Search and Custom Award download agency dropdowns so that Sub-Agency names that match the TopTier agency name are no longer returned.
  • Updated the Financial Assistance Award Summary pages to display all Assistance Listings/CFDAs associated with an award, rather than just the predominant CFDA:
    • When multiple CFDA apply, the CFDA overview at the top of the page shows the predominant CFDA, highlights the total number of programs involved, and provides a link that scrolls to the CFDA Program section for users seeking more detail.
    • CFDA Overview
    • When multiple CFDA apply, the CFDA Program section shows each and the funding it contributes. Both table and treemap views are available. Users can click on each CFDA to see more detail about the program.
    • CFDA Table
    • Added a column for 'CFDA Number' to the Transaction History tab of the Award History table so it is clear which CFDA is associated with each transaction. Also tightened the column widths within the table.
    • New CFDA Column
    • The screenshots illustrating this update were taken from this grant from the Department of the Interior to the Menominee Indian Tribe of Wisconsin.

API

Updates and Fixes

  • Added CFDA Number to transactions endpoint.
  • Fixed data source for 'pop_country_name' in the materialized views for FPDS transactions and awards containing FPDS transactions.
  • For search and download endpoints that use the 'filters' key, the 'NAICS' filter now works differently:
    • While legacy functionality is still supported at present, the current NAICS filter functionality will support a NAICS filter object with two arrays.
    • When 'require' is provided, search will only return results that have a NAICS code that starts with one element from the require list.
    • When 'exclude' is provided, search will only return results that do NOT have a NAICS code that starts with any element from the exclude list.
    • If the same code is included in both lists, both rules apply (and no results will be returned).
    • If a prefix of a code in one list is provided in the other (e.g. exclude contains '11' but require contains '1111'), the more specific (longer) rule supersedes when it applies. In the previous example, '111123' would be returned, but '112223' would not, since it does not start with the '1111' provided in the require array.
    • If a code is provided to the NAICS filter with less than 6 characters, it will match any NAICS code with the provided characters as a prefix. For example, if '11' is provided, '111111', '112211', '112233', and so on will match. Codes now MUST be 2, 4, or 6 characters in length.

Data and Download Notes

  • Improved our sub-award linking algorithm and processes:
    • Aggregate prime awards are now excluded from the consideration pool when looking for a prime award to link a new subaward record to.
    • In the unlikely event that multiple prime awards match the linking criteria, no link is established.
  • Updated agency reference data, adding and modifying several agency URLs and CJs. This data affects the Agency Profile pages.

March 23, 2020

Website

Updates and Fixes

  • Advanced Search:
    • Updated 'Agency' filter autocomplete so that it only returns Awarding Agencies, Awarding Sub-Agencies, Funding Agencies, and Funding Sub-Agencies that are actually present in the data; if an agency is not listed in the underlying transactional award data, it will not show up here. This change was made so that users aren't presented with selections upfront that can never lead to search results.
    • Fixed a bug of recent vintage that prevented some Advanced Search URLs from properly replicating the locked-in search filters when pasted in a new tab or shared.
    • Updated the search results Table so that it properly displays $0 amounts as $0 instead of blank.
    • Fixed a bug with how the 'Award Amounts' filter interfaces with loan awards.
    • In the 'Agency' filter: in a few cases, Sub-Agency names were repeated multiple times in the autocomplete for the same TopTier Agency. We updated the autocomplete behavior so that the same Sub-Agency name now only displays once per TopTier Agency. This has no effect on search results.
    • Visit our Advanced Search page to view these updates
  • Award Summary Pages:
    • Fixed a bug so that Recipient addresses are correctly and completely displayed at the top of the page.
    • Improved the consistency, clarity of labels, and completeness of the Parent Award Details and Related Awards sections of Contract and IDV Award Summary pages.
    • A very small number of agencies have reported award data to FPDS or FABS but do not have agency profile pages, since they do not submit full quarterly DATA Act submissions to Treasury. Up to this point, award summary pages from awards tied to these agencies contained broken links to those agencies' non-existent agency pages. We removed the broken hyperlinks.
  • Fixed a bug related to the rollover to FY20 account data that affected the visual display of the Fiscal Year dropdown in the Spending Explorer pages
  • Fixed a bug of recent vintage that limited the display of results when searching for multiple DUNS at one time (separated by spaces) in the Keyword Search page

API

Updates and Fixes

  • Bulk Download endpoints now take type codes instead of type names for prime award types and sub award types. See here for more details.

Data and Download Notes

  • Custom Award Data Download page:
    • Combined the 'Award Level' and 'Award Types' sections. Users can now more clearly choose the exact award types (including Sub-Award types) they want included in the download. They also may choose Insurance for the first time as a separate category.
    • New Award Type Selector
    • Updated the 'Location' section to allows users to choose between Recipient Location and Place of Performance (previously, only Recipient Location was on offer).
    • New PPoP Location Filter
    • In the 'Agency' section: in a few cases, Sub-Agency names were repeated multiple times in the SUB-AGENCY box for the same TopTier Agency. Now they will only display once. This has no effect on the downloads generated.
    • Visit our Custom Award Data Download page to view these updates
  • Derived fields primary_place_of_performance_scope and prime_award_primary_place_of_performance_scope were not properly filled in for a subset of data; updated the data to fill it in for all remaining records.

March 09, 2020

Note for Delta File Users: February/March Delta Files

The February delta files for All Financial Assistance and All Contracts did not include every record that they should have. We have incorporated all records that were missing in the February deltas within the March deltas; once you integrate the March deltas, you should be back up to date. If you do not already use the delta files, none of this will have an impact on you. Delta files are located here.

Website

Updates and Fixes

  • Added links at the bottom-right of every page to our social media platforms.
  • Added social share buttons to our Glossary entries, including the ability to easily link them.
  • Fixed a bug that caused inconsistencies in the 'Unreported Data' field of Spending Explorer between 'Budget Function' and 'Object Class' starting points.
  • Fixed a bug that prevented the federal account dropdown menu from displaying more than 100 federal accounts (e.g., for DOD) on Custom Account Download.
  • Updated the Custom Award Download agency dropdown to remove a couple small agencies with no award data, add a few small agencies that did not previously appear but that do have award data, and to consolidate duplicative SubTier name entries in the dropdown within all agencies that featured them.
  • Fixed a bug of recent vintage that prevented award transactions from appearing on certain assistance Award Summary pages when special characters appeared in the unique award key.

API

Updates and Fixes

Data and Download Notes

  • The spending endpoint now provides consistent unreported data amounts between type: budget_function and type: object_class.
  • The spending by award and spending by award count no longer accept recipient_id as a filter.
  • If a filter is added that an endpoint can't use, an extra message is added to the messages array from the response stating that it was not used.
  • Elasticsearch version upgraded from 6.2 to 7.1.

February 24, 2020

API

Updates and Fixes

Data and Download Notes

  • Added a new column to the beginning of all Custom Account Data Download files called reporting_agency_name. This column provides the name of the agency that submitted and certified that row of account data in the DATA Act Broker. Sometimes this agency differs from the owner of the TAS itself (as captured in the agency_identifier and agency_name fields).
  • Updated the 'File Type' section of Custom Account Data Download so that multiple types can be selected per download, rather than only one of the three. The related api/v2/download/accounts endpoint was also updated to service multiple types at once. When multiple types are selected, they are bundled into a single zip file.
  • Updated federal account bucketing so that certain AID = 011 Federal Accounts are now grouped under the Department of Defense throughout the site. Additional details can be found here.
  • The API responses for Spending By Category and Spending By Geography endpoints for counties are now grouped by county code instead of name + code to improve consolidation. Also updated reference data of counties and states (used for cleaner aggregations and API responses), which now updates automatically from https://geonames.usgs.gov/docs/federalcodes/NationalFedCodes.zip.
  • Updated the Custom Account Data Download agency dropdown list to show only those agencies that report quarterly to the DATA Act Broker. In so doing, also added two agencies to this list that were previously missing.
  • Fixed a bug of recent vintage on the download waiting pages that could lead to flashing loops.
  • Upper-cased all program activity names to prevent them from being treated as distinct program activities based on casing differences alone and to normalize display.
  • Fixed a bug in the DABS ETL script where a sufficiently large certified DABS submission in Broker could cause an error during the load into USAspending.gov's database. The load_submission management command was updated to chunk File C during processing instead of loading the entire file into memory.

February 10, 2020

Website

Updates and Fixes

  • Improved the Recipient and State profile pages so that they update their URLs when the user selects a Time Period filter (e.g., FY18). Previously, these selections did not change the URL and thus links shared to this page did not capture the specific time frame that the link sharer was viewing.
  • Fixed a bug that prevented the Federal Account Profile pages from loading properly on tablet and mobile devices.
  • Fixed a bug in IDV Award Summary Pages where the "This IDV" tab under the "Award Amounts" section was displaying blank data.
  • Fixed a bug that prevented users from typing within Advanced Search filter boxes while the Glossary was open.
  • Fixed a bug that caused the Featured Content section of the homepage to display improperly on Internet Explorer
  • Updated the name of the 'Broadcasting Board of Governors (BBG)' to 'U.S. Agency for Global Media (USAGM)' throughout the site to reflect its name change.

API

Updates and Fixes

  • Upgraded application from Django 1.11 to 2.2, a significant update to keep the site current and secure.
  • Created new transaction data tables source_assistance_transaction and source_procurement_transaction, which are exact Schema copies of the published_award_financial_assistance and detached_award_procurement in the Data Act Broker database, respectively.
  • Removed references_location table, which simplifies the Schema and reduces database volume. All data were originally sourced from transaction_fabs and transaction_fpds and can still be found there.
  • Updated api/v2/references/naics response so the count value is now 0 instead of 1 for NAICS codes with no children.
  • Updated on-screen version of api/v2/download/status/ and api/v2/bulk_download/status/ so that they correctly display as GET endpoints (the endpoints themselves were already GET, this was just a display issue).
  • Added more detailed response to download count endpoint api/v2/download/count/ to provide more context of the download limit and explain invalid requests.
  • Updated API documentation for api/v2/search/spending_by_award/ endpoint to state that award_type_codes is a required field. The endpoint itself already required this field; this merely aligned the documentation with the existing function.

Data and Download Notes

  • In order to align with name changes made in the Federal Procurement Data System (FPDS), updated the following column headers for all Contract Award data downloads (as well as the "Business Types" field of the contract award summary pages)
    • gfe_gfp_code becomes government_furnished_property_code
    • gfe_gfp becomes government_furnished_property
    • alaskan_native_owned_corporation_or_firm becomes 'alaskan_native_corporation_owned_firm_
    • foreign_owned_and_located becomes foreign_owned
    • native_hawaiian_owned_business becomes native_hawaiian_organization_owned_firm
    • tribally_owned_business becomes tribally_owned_firm
    • receives_contracts_and_grants becomes receives_contracts_and_financial_assistance
    • receives_grants becomes receives_financial_assistance
  • Fixed a bug where subawardee_business_types was being transmitted in the sub-grant downloads as two-character codes instead of the names corresponding to those codes.

January 27, 2020

Website

Updates and Fixes

  • Award Summary Pages
    • Added recipient address to the header section of the site so it can be quickly viewed as soon as the page is loaded. A special version is displayed for awards with aggregated recipients or recipients that have been redacted to protect personally-identifiable information (PII) of individuals. The corresponding glossary entries for these special recipient types were also updated.
    • Recipient Example
    • Multiple Recipient Example
    • Redacted PII Example
    • Added the NAICS/PSC hierarchy and an improved tooltip to the Description section of Contract IDV pages. It had previously been added to Contract pages.
    • Check out these updates in this award from the Department of Energy to the Regents of the University of California to fund the Lawrence Berkeley National Laboratory
  • Updated Award Summary Page URLs to properly URL-encode characters that are in the unique award key but are non-standard for URLs (e.g., spaces or $ ^ @ / ' # % Â or ! characters). With this update, these characters are now properly URL-encoded in the usaspending_permalink field in the download and the URLs created when users navigate to affected award summary pages on the site. The unique award keys themselves are unchanged and continue to include the original characters. This preserves the original unique_award_key data, the direct link between them and the Award Summary Page URLs, and the functionality of those URLs when special characters are involved.
    • Example of non-functional old URL with non-standard characters:
      • usaspending.gov/#/award/ASST_NON_R!D1102A37 10_12E2
    • Functional URL for same page after URL-encoding non-standard characters:
      • usaspending.gov/#/award/ASST_NON_R%21D1102A37%20%20%20%2010_12E2/
  • Advanced Search
    • Updated the 'Award ID' filter such that it now accepts quotation marks that, when provided, force a search for the entire string as one unit matched against the entire Award ID. This affects Award IDs that contain characters that are interpreted as delimiters (most commonly: spaces, -, or _ characters), whose components continue to be searched like separate 'words in a sentence' if they are not wrapped in quotation marks.
      • For example, compare the following Award ID searches:
        • "RRB_201902_ 8051_ AL*003" yields 1 result
        • RRB_201902_ 8051_ AL*003 yields 1 result
        • RRB_201902 yields 11,020 results
        • AL**003 yields 155 results
        • "AL**003" yields 0 results
        • 0001 yields 230 grant Award IDs that contain "0001" such as FR-INF-0001-17
        • "0001" yields 0 grant Award IDs, since none of these Award IDs solely consist of 0001
    • Fixed a bug in the link from the Sub-Awards ID field in Advanced Search. Previously, the link would open the relevant Award Summary Page but did not always properly scroll to the Sub-Award section of the page.
    • Fixed a bug that prevented users of Internet Explorer or Microsoft Edge from using the Program Source (TAS) filter successfully.

API

Updates and Fixes

  • Improved the API response when requesting downloads to reduce confusion and point API users to the relevant status endpoint to monitor download generation progress.
  • Improved messages for success and error responses for those API endpoints limited by design to only surface award data from FY2008-present.
  • Refactored code so now-redundant legal_entity table could be removed from database, reducing database size significantly.
  • Improved queue classes and functionality which are a core component of the download generation process.
  • Optimized unit and integration tests, significantly reducing runtime for test executions.
  • Squashed database schema definition files to simplify database initializations, which is a common activity during code tests.

Data and Download Notes

  • Account Data
    • Added 'last_modified_date' to all Account Data downloads. This field conveys the date a given row of data was published.
    • For Account Breakdown by Award downloads throughout the site:
      • For clarity and consistency across files, renamed:
        • 'piid' to 'award_id_piid'
        • 'parent_award_piid' to 'parent_award_id_piid'
        • 'fain' to 'award_id_fain'
        • 'uri' to 'award_id_uri'
        • 'primary_place_of_performance_zip_4' to 'primary_place_of_performance_zip_code'
      • To add value and improve consistency across files (note that all of these fields require that the Account Breakdown By Award data in the row has been successfully linked to an Award):
        • Added 'award_unique_key'.
        • Added 'period_of_performance_start_date', 'period_of_performance_current_end_date', and 'ordering_period_end_date'.
        • Added 'awarding_office_code', 'awarding_office_name', 'funding_agency_code', 'funding_agency_name', 'funding_sub_agency_code', 'funding_sub_agency_name', 'funding_office_code', and 'funding_office_name'.
        • Added 'product_or_service_code' and 'product_or_service_code_description.'
        • Added 'usaspending_permalink'. For Account Breakdown By Award data that has been successfully linked to an Award, this contains the URL of its Award Summary Page.
        • Added 'recipient_zip_code' and 'primary_place_of_performance_zip_code' to the Federal Account version of the file, where they had been missing.
    • Improved how federal account level data is collated in regard to direct_or_reimbursable_funding_source and eliminated spurious rows ($0 across all amount fields) where possible from generated downloads for Account Breakdown by Program Activity & Object Class and Account Breakdown by Award files.
    • Updated 'federal_account_name' to use the title of the newest TAS within the Federal Account. This change applies to all Account Data downloads and Federal Account Profiles.
  • Award Data
    • For all assistance downloads (transaction and award level)
      • Added 'primary_place_of_performance_scope' to reduce confusion surrounding the place of performance fields in assistance data. It clearly conveys the range of the place of performance for a given award, from the narrowest (Single ZIP code) to widest (Multi-state or Foreign Country).
    • For transaction-level assistance downloads (not award-level) throughout the site, made the following changes for clarity and consistency across files, and to add value:
      • Renamed 'original_subsidy_cost' to 'original_loan_subsidy_cost'
      • Renamed 'total_subsidy_cost' to 'total_loan_subsidy_cost'
      • Renamed 'total_loan_value' to 'total_face_value_of_loan'
      • Added 'total_obligated_amount'
      • Added 'total_non_federal_funding_amount'
      • Removed 'total_funding_amount' entirely because it meant something quite different at the transaction level than the award level and contributed to significant confusion as a result. As merely the sum of 'federal_action_obligation' and 'non_federal_funding_amount', it also added little value.
    • For all contract downloads (transaction and award level) throughout the site:
      • Renamed 'parent_award_id' to 'parent_award_id_piid' for consistency.
    • For award-level contract downloads (not transaction-level) throughout the site:
      • Added 'price_evaluation_adjustment_preference_percent_difference' for consistency with the transaction-level file.
  • Sub-Grant Data
    • Added 'prime_award_primary_place_of_performance_scope' to all sub-grant data throughout the site.
  • Custom Award Download:
    • Added a TXT (Pipe-delimited) file format option. Downloads are .txt files with '|' characters separating values in the same row and new lines separating rows. The two existing file format options remain unchanged.
    • Improved SQL query coding in Custom Award Download so that downloads complete faster.

January 06, 2020

ZIP File and Filename Changes

In the January 6th deploy, we are changing the naming conventions of downloaded ZIP files and files contained within those ZIPs. The new naming conventions are more internally consistent, legible, information-rich, and perhaps most importantly, include a timestamp on all files; this last change means that you will no longer run into issues concurrently opening two files with the same name that came from separate downloads. See this XLSX attachment for a complete list of the new names. Note: we changed the formatting of one of the Custom Account downloads after some internal testing. These two changes are highlighted in yellow in the attached spreadsheet.

Website

Updates and Fixes

  • Updated all internal links to Award Summary Pages throughout the site so they serve up a permanent URL containing the unique award key rather than the old internal database ID format.
  • Improved the Dates section on the Award Summary Pages 2.0 pages. New features include color coding for date types; coordination of the 'Today' position marker versus end date markers; and a current status indicator positioned directly above this section. Example screenshots:
  • Added hyperlinks from the Sub-Award ID column in Advanced Search to the relevant prime award page. When you click this link, the site opens that prime award page, scrolls down to the Award History section, and opens the sub-award tab.
    • View an example of this update by performing an Advanced Search, toggling the Sub-Awards button, and clicking on any field in the Sub-Award ID column.
  • Updated interior page titles so that they reflect the content of the page. These titles appear as the tab name in web browsers and in search engine results.

Data and Download Notes

  • Added three new columns to sub-award downloads throughout the site:
    • subaward_fsrs_report_last_modified_date
      • subcontract and subgrant
      • date the report that contains the subaward data was last updated in FSRS.
    • subaward_fsrs_report_id
      • subcontract and subgrant
      • allows those with appropriate FSRS access to directly link to the FSRS report that led to this subaward record in USAspending. For example, subaward_fsrs_report_id = 2975c5062fdc0ecaa2c16c4dd4fab03b is used directly in an FSRS URL to tie back to the source report
    • prime_award_project_title
      • subcontract only
      • information on the project entered by the prime contractor into FSRS
  • Updated the titles for two existing columns in sub-award downloads throughout the site for clarity:
    • subaward_report_year is now subaward_fsrs_report_year
    • subaward_report_month is now subaward_fsrs_report_month
  • Updated submission_period column in Account Breakdown by Award download (Custom Account Data) and Award Summary Page download (Federal Account Funding file). This field is now formatted as "FY" + #### + "Q" + # (e.g., FY2019Q3 for Fiscal Year 2019 Quarter 3). Previously it just listed the last day of the submission period.
  • Updated download status page title and text to more accurately describe the status of download file generation, particularly for downloads that failed to generate. (Prior to these changes, failed downloads were described as 'in progress' on this page.)

December 16, 2019

Upcoming ZIP File and Filename Changes

In the January 6th deploy, we will change the naming conventions of downloaded ZIP files and files contained within those ZIPs. The new naming conventions are more internally consistent, legible, information-rich, and perhaps most importantly, include a timestamp on all files; this last change means that you will no longer run into issues concurrently opening two files with the same name that came from separate downloads. In order to ease the transition for all those who have built scripts based on the old names, we are providing the new nomenclature in advance. See this attachment.

Website

Updates and Fixes

  • Advanced Search:
    • In the Spending by Sub-Award table, entries under the Prime Award ID column now link to the award summary page of the prime award the sub-award is attached to.
    • PrimeAward Links example
    • View this update by performing an Advanced Search, clicking the Sub-Awards toggle in the top right above the table, and scrolling to the Prime Award ID column.
  • Custom Award Data Download
    • Enabled the Tab-Separated Value (TSV) file format for downloading data. When selected, it generates a .tsv file. Users may find this a helpful alternative in cases where they've run into trouble loading a .csv that has commas embedded within text fields.
  • Custom Account Data Download
    • Fixed a bug with the 'Account Breakdown by Program Activity & Object Class' download that prevented the full set of expected data rows from surfacing much of the time due to not properly accounting for the direct_or_reimbursable_funding_source field in the criteria used to determine a unique data row.
    • Updated the Federal Accounts listed under Department of Defense to include those from Army (017), Navy (021), and Air Force (057).
  • Award Summary Pages:
    • Fixed a bug that caused tooltips to display improperly on Internet Explorer.

API

Updates and Fixes

  • Added county_code values to location data returned by api/v2/awards/.
  • New capability to serve data downloads in Tab Separated Values (TSV) format in addition to the default CSV format.

Data and Download Content Notes

  • Added a usaspending_permalink header to all award and subaward data downloads that contains a hyperlink to the relevant Award Summary Page. This link will work to navigate you to that page in perpetuity.
  • Added last_reported_submission_period to account data from the 'Account Balances' and 'Account Breakdown by Program Activity & Object Class' downloads. This header identifies the fiscal year and quarter of the submission that the row of data originated in.
    • View this update by downloading 'Account Balances' or 'Account Breakdown by Program Activity & Object Class' file types on our Custom Account Data Download page.
  • Updated direct_or_reimbursable_funding_source to contain the values 'D' and 'R' instead of '1' and '2' in all account data throughout the site and downloads to make the meaning of the codes more obvious.
  • Updated treatment of the object_class_code field in all account data throughout the site and downloads so that the Unknown object class code is consistently conveyed as '000' instead of a mix of '0' and '000'.
  • Removed annual_revenue and number_of_employees headers from all award data downloads as planned and telegraphed in the July 15, 2019 release notes. This was due to the 2018 removal of these fields from our contract data source, the FPDS Atom feed.

December 02, 2019

Website

Updates and Fixes

  • Fixed a bug which caused the download button and tooltips on Award Summary pages to not display properly in Internet Explorer.
  • Fixed a bug that caused a small number of Award ID links in Keyword Search to go stale.

API

Updates and Fixes

  • Upgraded target Python version from 3.5 to 3.7 in preparation for Python 3.5's scheduled EOL in 2020. Backward compatibility to older python interpreters will not be maintained.
  • Significantly refactored the FPDS ETL process to improve handling of source data and increase performance.
  • Additional v1 endpoints were deprecated. See API endpoints for the current list.
  • /api/v2/search/spending_by_award and /api/v2/search/spending_by_award_count endpoints now support object_class and program_activity filters.
  • All API endpoints now return the deterministic generated award id generated_unique_award_id/unique_award_key for easier linking to award summary pages.
  • Refactored frontend to use v2/awards endpoints instead of v1/awards endpoints.

November 18, 2019

Website

Updates and Fixes

  • Award Summary Pages
    • Added a CFDA Program / Assistance Listing section to all financial assistance pages. This section includes a wealth of information sourced from beta.SAM.gov on the predominant CFDA funding the award, including its objectives, website and beta.SAM.gov page links, and eligibility criteria.
    • Award Date Ranges
    • Added remaining explanatory tooltips across all pages and updated some existing tooltips to display newly-revised definitions and explanations.
    • Updated Award History section to display counts for Transaction History and Federal Account Funding tabs on the IDV page. On grants and contract pages, updated Sub-Awards tab with improved language and fixed a bug where the Sub-Award Percentage of Prime Award Obligated Amount was miscalculated in some cases.
    • Updated Additional Information section for contract pages to properly fill out the Parent Award Details.
    • Minor improvements to the labeling of the date section of financial assistance pages and the spacing of the Description section

API

Updates and Fixes

  • Improved /api/v2/awards/<award_id>/
    • Return more parent award details in the contract award response.
    • Added record_type, applicant_eligibility, and beneficiary_eligibility to the assistance award response.
    • Added NAICS and PSC hierarchy information to the IDV response.
  • Transaction ETL logic now utilizes the generated transaction_unique_id for determinig earliest and latest transactions on an award when action_date and modification_number are not sufficient to narrow things down to a single transaction.
    • This field takes into account FPDS's transaction_number field which is common for procurement records from DoD.
  • Refactored TAS ETL script to store internal_end_date and internal_start_date in the database and unified the CSV and Broker-sourced loaders.
  • More API endpoints return the deterministic generated award id generated_unique_award_id/unique_award_key for easier linking to award summary pages.

Data and Download Notes

  • In assistance data downloads throughout the site, changed
    • (award summary level only) obligated_amount -> total_obligated_amount to highlight that this is a sum total across all the award’s transactions.
    • (award summary level only) non_federal_funding_amount -> total_non_federal_funding_amount to highlight that this is a sum total across all the award’s transactions.
    • (award summary level only) total_subsidy_cost -> total_loan_subsidy_cost to highlight that this is a field applicable to loans only.
    • (award summary level only) total_loan_value -> total_face_value_of_loan to highlight that this is a field applicable to loans only.
    • (award summary level only) Removed face_value_of_loan, original_subsidy_cost because they are transaction-level not award summary-level fields.
  • In contract data downloads throughout the site, changed
    • commercial_item_test_program_code -> simplified_procedures_for_certain_commercial_items_code to match the same change in FPDS
    • commercial_item_test_program -> simplified_procedures_for_certain_commercial_items to match the same change in FPDS.
    • (award summary level only) obligated_amount -> total_obligated_amount to highlight that this is a sum total across all the award’s transactions.
  • In account data downloads throughout the site, improved how agency_name and allocation_transfer_agency_name are populated; they are now based on agency_identifier and allocation_transfer_agency_identifier exclusively, and are never blank.

November 04, 2019

Website

Updates and Fixes

  • Award Summary Pages v2.0 Launch: updated Award Summary Pages to v2.0 and retired v1.0. V2.0 offers a number of improvements, including (but not limited to):
    • Page URL: the URL is permanent and uses the unique award key rather than the database ID integer. This allows you to move back and forth from downloaded data to the associated Award Summary Page in one step. Note that we will continue to accept the old Award Summary URL format.
    • Download: added a download button. The download includes all source data for the page (including award transactions, sub-awards, and federal account data) across several CSV files, along with a text file describing the contents of each file and a copy of the data dictionary.
    • Page-wide changes:
      • Added live links to agency profile pages, recipient profile pages, and the parent award page wherever appropriate.
      • Added extensive hoverable tooltips, which add background information and context helpful to understand the information on the page and how data is being displayed.
    • Header: added a header section that summarizes key information about the award in one place. Includes a timeline visualization of dates and the amount of time remaining on the award.
    • Award Amount Section:
      • Improved visuals and added hover states and a link that scrolls to the transaction table.
      • Contracts: Updated to clearly distinguish between obligated amount, current award amount, and potential award amount. Added display for many edge cases.
      • Grants: Updated to visualize non-federal funding amount.
    • Description Section: for contracts, updated to add context to NAICS / PSC by displaying the full nested hierarchy of the codes attached to the award.
    • Federal Accounts Section: patterned after the version on the IDV pages, this new section displays the federal accounts that fund the award and includes a scrolling link to the federal account data tab. Users can choose between a table and treemap view.
    • Award History tabs: added counts, so you can see how many award transactions, Sub-Awards, and federal account funding transactions there are at a glance and prior to clicking the tabs.
    • Additional Information Section: improved formatting, individual collapsible sections, and new fields.
    • Coming Soon:
      • The CFDA Program/Assistance Listing Section on Assistance pages will provide context on the program the award was issued under: program objectives, the applicants and beneficiaries it is intended for, a link to the program website, and a link to the authoritative reference information on the program at sam.gov.
      • Contract and Grant Activity charts will visualize award modifications and their $ amounts on a timeline.
      • The remaining explanatory tooltips will be added to the page in two weeks.
  • Homepage: updated the homepage banner numbers for FY 2019. A much more detailed treatment of this information can be found in the refreshed 2019 Your Guide to America’s Finance on our partner site.
  • Agency Profile Pages: added pages for two small agencies: District of Columbia Courts and Patient-Centered Outcomes Research Trust Fund.

API

Updates and Fixes

  • Re-wrote existing API documentation to meet the API Blueprint specification. All API docs are now machine-readable and can be used for conformance tests to reduce regression bugs.
  • Added new endpoint to search for valid NAICS /api/v2/references/naics/.
  • Purged many v1 endpoints from the API as part of the planned lifecycle of the API. The endpoints removed will return a HTTP 410 response for continuity. See our API docs for the list of active endpoints.
  • Added a subset of the production database which can be helpful to developers running a local USAspending API application.
  • Refactored how agency data are stored in the database for easier linkage to other data sources (quarterly submission files and transactions).

Data and Download Notes

  • All Sub-Award data downloads:
    • Dropped prime_award_fiscal_year from all subaward downloads. It was duplicative of prime_award_base_action_date_fiscal_year.
    • Renamed prime_award_action_date_fiscal_year to prime_award_base_action_date_fiscal_year to be clearer that it represents the fiscal year of the earliest (base) prime award action.
    • Renamed prime_award_action_date to prime_award_base_action_date to be clearer that it represents the action date of the earliest (base) prime award action.
  • Prime Contract Data:
    • FPDS.gov (the source of our contract data) has altered their treatment of period_of_performance_start_date (called Effective Date in FPDS). For awards with action_date on or after October 25, 2019, the period_of_performance_start_date will now propagate from the base award (modification_number = 0) instead of allowing each mod to have its own period_of_performance_start_date. Data with action_date prior to this point will not be altered. For more information, see here.

October 21, 2019

Website

Updates and Fixes

  • Site-Wide: updated grouping logic for Department of Defense (DOD) to unroll Corps of Engineers – Civil Works (USACE) from DOD throughout the site. Specific affected areas include:
    • DOD $ totals and drilldown contents in the Spending Explorer will no longer include USACE. USACE is separately broken out.
    • Spending by Category in Advanced Search will now break out DOD and USACE.
    • Downloads in award data archive, custom award download or custom account download where DOD is used as an agency filter will no longer include USACE data. Next sprint, we are adding USACE as a selectable agency filter option in these three downloads.
  • Sub-awards: fixed a bug that duplicated sub-award data on the site whenever we (infrequently) fully reloaded subaward data, and removed all duplicates created through previous reloads.
  • Corrected a bug of recent vintage in the sub-award downloads where prime_award_funding_agency_code and prime_award_funding_agency_name were mapped to the wrong database fields. They are now mapped correctly.

API

Updates and Fixes

October 07, 2019

Website

Updates and Fixes

  • Advanced Search:
    • Award Amount filter has been updated to fix a bug with searches involving $0 and to make the $ range search and checkbox search consistent with each other.
    • Time Period filter has been updated to surface results in the award table whenever the selected time period(s) intersect the earliest or latest action date under the award (looking at all its transactions). Previously, it only looked at the latest action date under the award.
    • To illustrate: under the previous system, only awards A1, A2, and A4 would surface in the table. Under the new system, awards A3, A5, and A8 will also surface.
      • Award Date Ranges
    • Links to Prime Recipient profiles have been added throughout the Advanced Search Tables and Categories tabs.
    • Fixed a bug with the Keyword Search filter that involved redundant double-tagging of active keyword search terms.
  • IDV Award Summary Page: IDV Activity Chart now displays all awards visually regardless of how short the start and end date range is.

API

Updates and Fixes

  • Fixed a bug that resulted in errors in API calls run through the default Django Rest Framework UI in non-local environments.
  • Added several new endpoints to provide additional award details. Further details are provided on the API documentation site:
  • Improved accuracy when searching for awards within date range(s)--discussed above.
  • Added support for 2 and 4-digit NAICS codes.
  • Added parent_generated_unique_award_id to the procurement award response for /api/v2/awards//, allowing for persistent linking to parent IDVs.
  • Updated solicitation_date provided in downloads to type Date instead of Datetime for a cleaner display.

Data and Download Notes

  • Custom Account Download: fixed a bug where certain agency_name data was erroneously appearing blank.
  • Award data downloads:
    • Updated award-level downloads site-wide to use the award_description from the base transaction rather than from the latest transaction.
    • Updated award-level downloads site-wide to use the most recent non-null executive compensation data, if any exists.

September 20, 2019

Website

Updates and Fixes

  • Added unique award key to contract and assistance data downloads throughout the site. This has been a consistent request of power users of the data, but can help all users understand how transactions relate to one another. All transactions that share a unique award key are rolled up for presentational purposes using that key (e.g., every transaction on a given Award Summary page shares the same unique award key). While the site has always used an internal unique award key, we are now surfacing it. See Data and Download notes below for more information, including format.
  • Added treasury_account_name to Custom Account Download for all TAS-level downloads. This field provides the name of the treasury_account_symbol field, which provides useful information on the purpose of the treasury account.
  • Fixed a bug of recent vintage that had disabled a workaround to the FSRS practice of stripping out hyphens in subaward data. With the workaround restored, Prime Awards that contain hyphens in the award_id_fain (FAIN) field will now properly link to their subawards again.
  • Advanced Search: improved Sub-Account Code tooltip explanation within the TAS search filter.
  • IDV Summary Page: improved hover tooltip on the IDV Activity Chart to distinguish between parent and grandparent IDVs.
  • Glossary: Clarified entry for Sub-Account code; updated entry for TAS to include link to definition for Federal Account.

API

Updates and Fixes

Data and Download Notes

  • Added unique award keys to contract and assistance downloads throughout the site. They are created from a concatenation of several fields in each record. If a component of the unique award key isn’t present in a given record, then "-NONE-" is used in place of that component in the concatenated string. Additional formatting information:
Award Type Unique Key Element Name Concatenation of Elements Example
Prime Contract contract_award_unique_key "CONT_AWD"; award_id_piid; agency_identifier; parent_award_id; parent_award_agency_id CONT_AWD_TDOX11F0020_2001_GS23F0187M_4730
Prime Contract IDV contract_award_unique_key "CONT_IDV"; award_id_piid; agency_identifier CONT_IDV_TOS15F0019_2001
Prime Financial Assistance Aggregate assistance_award_unique_key "ASST_AGG"; award_id_uri; awarding_sub_agency_code ASST_AGG_15CA35050692501_12F2
Prime Financial Assistance Non- Aggregate assistance_award_unique_key "ASST_NON"; FAIN; awarding_sub_agency_code ASST_NON_1805CA5MAP_7530
  • Notes:
    • Note 1: an aggregate assistance record includes record_type = 1. A non-aggregate record includes record_type <> 1.
    • Note 2: agency_identifier is an FPDS field capturing the agency that submitted the record to FPDS (often distinct from the awarding or funding agency). We hold it in our database but do not surface it in downloads as a standalone field to avoid confusion with the other agency fields.

September 9, 2019

Website

Updates and Fixes

  • Sub-Award Downloads have been substantially improved throughout the website (see Data and Download notes below for more information).
  • Advanced Search: enhanced performance of TAS/FA filter in edge cases with massive numbers of returned search results; changed AvailabilityTypeCode component of TAS filter to be case-insensitive; fixed a minor bug where mouse click on greyed-out down carets in the location filter errored the page; separated BPOA/EPOA filter tags with a '/' instead of '-' in locked-in filter visual.
  • IDV Page: removed "Federal Accounts" label above treemap view to reduce redundancy and provide a cleaner user experience; fixed a bug where links to non-existent recipient profiles were being created on pre-FY2008 records.
  • About Page: updated link to Federal Government Procurement Data Quality Summary to point to the new FY18 version of the report.

API

Updates and Fixes

  • Performance improvements to the new TAS and Federal Account filtering in Advanced Search.
  • Subaward downloads are comprised of DAIMS 1.3.1 data and contain updated header.
  • Added solicitation date to different downloads.
  • Improvements to api/v2/awards/<award_id>/.
  • Running a mapping check between state code and state name to provide missing values when possible.
  • Display all CFDA related to an Assistance Award.

Data and Download Notes

  • Sub-Award data downloads throughout the site have been substantially improved in accordance with the changes from the DATA Act Information Model Schema v1.3.1 (see File F tab here). In addition to being presented in a more logical order, many new elements have been added, including (but not limited to):
    • Prime Award
      • Award Description
      • Base Action Date
      • Funding and Awarding Agency
      • NAICS / CFDA
      • Place of Performance
      • Prime Awardee Location
      • TAS and Federal Account
      • Unique Award Key
    • Sub-Award
      • Executive Compensation
      • Subawardee Business Types
      • ZIP Codes
    • Solicitation Date (solicitation_date in the download) has been added to Contract Award downloads throughout the site.

August 26, 2019

Website

Updates and Fixes

  • TAS & Federal Account in Advanced Search and Downloads: Added TAS and Federal Account (sometimes called “Program Source”) to Advanced Search filters and Award data downloads throughout the site. This was a significant technical undertaking that utilizes linked Award-level account spending data from agency financial systems submitted under the DATA Act. This change is a significant step forward in delivering on the promise of the DATA Act to link the Appropriation and Finance realms to Federal Awards. In coming sprints, we will add additional features to TAS/Federal Account search, including searching by title and code and enhanced discoverability and performance. See the Note in the Data and Download section for additional information on this data and its limitations.
  • IDV Summary Page: Added the completed IDV Activity module to the IDV Summary pages. This module provides a powerful, versatile visual overview of child and grandchild award orders under an IDV and their obligated amounts, periods of performance, and % of ceiling obligated. Users can select how many awards to view on screen at a time and see additional information by hovering over each award.
  • City Search: Fixed a bug where foreign locations with aberrant data in country code fields were not surfacing correctly via City Search; Fixed a bug where leading or trailing spaces were not being removed from city names in the autocomplete list.
  • Contract Summary Pages: Updated to source the Award Description from the initial transaction under the award, rather than the latest transaction.
  • Advanced Search: Fixed a bug where in certain edge cases, reloading a previously-searched URL hash would result in a blank screen; Fixed a bug where Sub-Grants connected to non-Grant assistance types displayed a count but no results in the Sub-Award table.
  • Glossary: Updated entry for TAS

API

Updates and Fixes

Data and Download Notes

  • TAS and Federal Account have been added to Award downloads throughout the site. Fields are called federal_accounts_funding_this_award and treasury_accounts_funding_this_award and take this format:

    • federal_accounts_funding_this_award:
    [Account 1];[Account 2];[Account 3]; ... [Account N]
Where N is the total number of accounts associated with the award in Custom Account Data, and [Account] has format
[AID]-[MAC]
    Example: "075-0512;075-0580;075-8004"
    • treasury_accounts_funding_this_award:
    [TAS 1];[TAS 2];[TAS 3]; ... [TAS N]
Where N is the total number of TAS associated with the award in Custom Account Data, and [TAS] has format
[ATA, if applicable]-[AID]-[X or BPOA/EPOA]-[MAC]-[SUB]
    Example: "012-075-2019/2019-0512-000;075-X-0512-000;075-2017/2018-0512-000;075-X-0580-000;075-2019/2021-8004-000"

    For more information on the TAS components, see here.

Note: This filter uses Account Breakdown by Award data (available here in full) submitted by agencies to Treasury under the requirements of the DATA Act of 2014, which went into effect in FY17Q2. As such, this data (and thus this filter) only covers award transactions from January 2017 onward. Awards that began prior to that point will only surface via this filter if they have financial modifications post-January 2017. Note that a subset of agency-submitted Account Breakdown by Award data is not definitively linkable to a single Federal Award; unlinked data cannot be and is not used by this filter, but is available along with the rest of the Account Breakdown by Award Data in the Custom Account Data section.

  • Action Date Fiscal Year: Action Date Fiscal Year has been added to added to all Prime Award and Sub-Award downloads throughout the site. New fields are called:
    • action_date_fiscal_year (Prime Award download)
    • prime_award_action_date_fiscal_year (Sub-Award download), and
    • subaward_action_date_fiscal_year (Sub-Award download). This is a system-derived field that provides the relevant Federal Fiscal year (which runs 10/1 to 9/30) based on the Action Date fields.
  • Executive Compensation: While it was already available on the Award Summary pages, Executive Compensation information sourced from SAM.gov has been added to Award downloads throughout the site. Only recipients that meet certain criteria are required to submit this information; as such, it is only available for a relatively small subset of Awards. New fields are as follows:
    • highly_compensated_officer_1_name
    • highly_compensated_officer_1_amount
    • highly_compensated_officer_2_name
    • highly_compensated_officer_2_amount
    • highly_compensated_officer_3_name
    • highly_compensated_officer_3_amount
    • highly_compensated_officer_4_name
    • highly_compensated_officer_4_amount
    • highly_compensated_officer_5_name
    • highly_compensated_officer_5_amount

August 12, 2019

Website

Updates and Fixes

  • Advanced Search: continued backend and frontend work in preparation for adding Treasury Account Symbol (TAS) and Federal Account to Advanced Search; fixed an issue where for a small number of loan awards, the total obligated amount displayed in advanced search did not match the total original_loan_subsidy_cost from the transaction download.
  • About page: added a Licensing section to the About page.
  • Award Summary pages: tweaked to display the most recent non-null Executive Compensation data for transactions under the award, rather than the executive compensation data for the latest transaction.
  • IDV Summary page: tweaked the Award Amounts section so that it no longer attempts to graphically display awards when all award amount fields are $0.
  • Fixed an issue of recent provenance with the monthly delta files under Award Data Archive where data for deleted records was displayed in the wrong columns.

API

Updates and Fixes

  • IDV activity default sorted by the obligated amount.
  • Official glossary entry for Primary Place of Performance Address updated to make it consistent with standardized definition.
  • Executive officer details added to /api/v2/awards/ for assistance awards.
  • Python code style now standardized to Black.
  • CONTRIBUTING.md and LICENSE files added to the project.

July 29, 2019

Website

Updates and Fixes

  • Advanced Search: Backend work in preparation for adding Treasury Account Symbol (TAS) and Federal Account to Advanced Search
  • City Search: added a tooltip clarifying the sparseness of place of performance city data in the case of foreign places of performance; fixed a bug where commas embedded in city name text were not properly translated into search filters.
  • IDV Summary page: added a Recipient Details section to the Additional Information module, added a tooltip explaining the Federal Accounts module, continued work on the IDV Activity module (launching in 1-2 sprints).
  • Site-wide: fixed a bug in tables where the last row of data (particularly when a scrollbar is present) was sometimes cut off and difficult to scroll, view, or select; fixed a bug where the custom award download (transaction-level) and ‘spending over time’ advanced search graph sometimes gave different $ totals.
  • Updated About page to add a brief 'Development and Releases' section that includes a link to the release note repository.

API

Updates and Fixes

  • Seven new autocomplete endpoints for TAS and Federal Accounts
  • New CGAC/AID reference data table.
  • Added new flag to /api/v2/awards/idvs/activity/ to hide edge cases (negative awards or no end date)
  • Updated executive compensation data to be stored with transaction records.
  • Corrected SQL logic used for categorization of loan transactions which was important for accurate aggregations.

July 15, 2019

Website

Updates and Fixes

  • IDV Summary page: added a Recipient Details component to the Additional Information section; various labeling consistency and explanatory improvements to the page; began bundling a copy of the data dictionary with each download originating from the IDV page (to be expanded to other downloads in the future).
  • City Search (Advanced Search): improved how domestic cities are surfaced in search for older data; improved autocomplete result ordering to be more useful (now in order of the # of transactions per city); improved how foreign cities are filtered and displayed; improved how the user interface responds to selecting or deleting from the city typeahead box.
  • Advanced Search (Miscellaneous): fixed a bug where subsidy cost was double-counted for transactional level totals for certain loans
  • Award Summary pages: fixed a bug of recent vintage where the subaward table was failing to scroll properly when large number of subawards were present.

API

Updates and Fixes

  • Completely redesigned Sub-Award data ETL from source systems to obtain DAIMS 1.3.1 FSRS/Sub-Award data
  • Updated the reponse of /api/v2/awards/<award_id>:
    • Includes additional IDV parent and parent recipient details: agency_name (under the parent_award object), referenced_idv_agency_desc, parent_recipient_name, parent_recipient_hash
    • Business categories data were changed from column name strings to human-readable text
  • Added parent IDV details to /api/v2/awards/idvs/activity/
  • Data Dictionary XLSX file is now provided in zip file downloads from the IDV summary pages
  • Improved "City Search" autocomplete in Advanced Search by providing the results by prevelence in transaction data and including the country when searching for cities outside of the United States

Data Notes

  • On October 26, 2018, FPDS.gov (our underlying data source for contract award data at the time of writing) removed vendor Annual Revenue and Number of Employees elements from their public data feed, after making a legal determination that these elements are For Official Use Only (FOUO) and not intended for public display. Given this, our data for these two fields is increasingly stale, and we will be removing them from our data downloads in the coming months. If you have questions or concerns with this change, you can contact the GSA’s Integrated Award Environment, which operates FPDS.gov. You can find more information on this FPDS change here: https://www.fpds.gov/wiki/index.php/V1.5_SP_3.0.

June 28, 2019

Website

Updates and Fixes

  • Added unique transaction keys to contract and assistance transaction-level downloads throughout the site. This has been a consistent request from power users of the data.
    • Assistance records: assistance_transaction_unique_key is a concatenation of awarding_sub_agency_code, award_id_fain, award_id_uri, cfda_number, and modification_number, with a single underscore ('_') character inserted in between each. If a field is blank, it is recorded as "-NONE-". Example: 7200_72068519LA00002_-NONE-_98.001_0000
    • Contract Award records: A concatenation of agencyID, parent_award_agency_id, award_id_piid, modification_number, parent_award_id, and transaction_number, with a single underscore ('_') character inserted in between each. If a field is blank, it is recorded as "-NONE-". agencyID is an FPDS field that captures the SubTier Agency that submitted the transaction to FPDS (often distinct from the awarding agency). These same six fields are part of the unique key for contract awards in FPDS. Example: 1422_1422_INL15PD01092_P00003_INL15PC00159_0
    • Contract IDV records: the same format as Contract Award is used, but only agencyID, award_id_piid, and modification_number are filled in, even if additional fields in the key are present in the IDV record; the rest of the fields are recorded as "- NONE-" for unique key purposes, even when present. This follows the unique key model for contract IDVs in FPDS. Example: 8000_-NONE-_80LARC17C0008_P00019_-NONE-_-NONE-
  • IDV Summary page: added explanatory tooltip and fixed a mapping and labeling issue in the Orders Made Under this IDV section tables with Awarding Agency
  • Unified unique award key format between USAspending and DATA Act Broker
  • Advanced Search: fixed a bug of recent vintage whereby the download button was clickable in situations when it was intended to be disabled.

API

Updates and Fixes

  • Promotion of unique_award_key into the USAspending awards table to simplify integration with Broker award data and increase data integrity/quality by ensuring more accurate matching during nightly ETL processes.
  • Added awarding agency to the IDV awards endpoint.
  • Updated restock_exec_comp to pull from duns table in Broker to ensure better consistency between USAspending and Broker.
  • Backfilled new FABS unique transaction keys to matches changes made in Broker. In USAspending, these values are used to synchronize transactions between USAspending and Broker.
  • Enhanced FPDS nightly loader to allow reloading of specific transactions.

Data Notes

  • Beginning June 28, 2019, FPDS.gov (the source of contract data on USAspending at the time of writing)
    • is reducing the maximum character length of the Award Description (Description of Requirement) field to 250 characters from 4000 for new contract awards (existing awards will be unaffected). You can find more information on this change here: https://www.fpds.gov/wiki/index.php/V1.5_SP_4.0
    • is applying the same requirements to the Solicitation Date field to civilian agencies as apply to DOD, which will lead to it appearing in more records than at present for new awards from this point forward. More information here: https://www.fpds.gov/wiki/index.php/V1.5_SP_4.0

June 14, 2019

Website

Updates and Fixes

  • City Search: added City to the Location Filter (Recipient Location and Place of Performance) in Advanced Search, replete with autocomplete (based on city names in our dataset). This represented a significant technical effort. City search is fully functional, but there will be forthcoming tweaks (e.g., to the frontend user experience) in subsequent releases.
  • IDV Summary page: minor label changes to ensure consistency with standardized nomenclature; fixed a bug where Funding/Awarding Office were not being displayed properly in the Additional Information section; continued work on the final module on the page (IDV Activity).
  • Agency Congressional Justifications of Budget (CJ): Added CJ links for each agency to the Agency Profile landing page and individual pages, and a ‘featured content’ pointer on the homepage. Only added CJ links that resolve properly and have CJ content.
  • Fixed a small issue in advanced search related with loan spending visualizations, where award-level subsidy totals were being used instead of transaction level totals (loans only).

API

Updates and Fixes

June 4, 2019

Website

Updates and Fixes

  • IDV Summary page: changes to a few on-screen labels (including in associated glossary entries) to standardize terminology across the site and for clarity; added 'Awarding Agency' column to Federal Account Funding tab and 'Total Count of Awarding Agencies' to Federal Accounts module; backend work related to the upcoming IDV activity module.
  • Substantial backend work related to upcoming city search functionality in advanced search. This will launch on the frontend soon with additional backend optimizations and changes coming at that time as well.

API

Updates and Fixes

  • New city key for location filters in Advanced Search API endpoints
  • New endpoint for finding cities in transaction data: /api/v2/autocomplete/city/
  • Updates to /api/v2/awards/idv/accounts/ and /api/v2/awards/idv/funding_rollup/
  • Updates to the IDV glossary terms for consistent naming conventions

May 20, 2019

Website

Updates and Fixes

  • IDV Summary page: Completed the Federal Accounts module by adding a treemap and paginated table which shows all federal accounts involved in child award and grandchild award orders under the IDV (note: treemap only includes positive obligated amounts). Added information-rich explanations that trigger upon graph hover in the Awards that Reference this IDV section. Numerous updates, corrections, and improvements to the Additional Information section. Fixed the Total Count of Funding Agencies, which showed an overinflated count in certain edge cases.
  • Advanced Search: Award-level (as opposed to transaction-level, which was already correct) download now matches the start date in the advanced search results table. Before, it included the start date of the latest transaction. Now, it correctly includes the earliest start date in the series of transactions under the award, just like the search results table.
  • Updated Congressional District geo-json shape files powering USAspending maps (including on state profile pages and advanced search results) to account for updates to congressional district maps in recent years.

API

Updates and Fixes

  • New Docker compose file to create a local development and testing environment
  • Updated many API Contracts found here to meet expected syntax
  • Corrected a few static URLs incorrectly pointing to beta.usaspending.gov or using HTTP instead of HTTPS
  • Added new unique_award_key to transaction tables to serve as a foundation for future improvements
  • Endpoints being staged for removal will include a new header: X-API-Warn as the first step for notifying users of impending changes
  • Updated /api/v2/awards/ response for Contracts and IDVs to include more fields and to standardize the field names
  • Added additional agency fields to /api/v2/awards/idvs/accounts/

May 6, 2019

Website

Updates and Fixes

  • IDV Summary page: download functionality has been added to the IDV award summary page. The download includes all relevant data from the page across several CSV files, along with a text file describing the contents of each file. In the future, we will be rolling out similar download functionality to the other award pages.
  • IDV Summary page: page has been updated across the board to surface ‘grandchild’ awards in three-tier award relationships (e.g., Federal Supply Schedule <- BPA <- BPA Call) and to display the associated data and counts throughout the page.
  • IDV Summary page: fixed the mapping of the Federal Account Name column in the Federal Account Funding tab, which also corrected the Total Count of Federal Accounts in the Federal Account Module
  • Advanced Search: fixed a bug where subawards underneath IDVs were not being displayed properly in the subaward results table and fixed a few typos in existing advanced search filters.
  • Keyword Search: added a Contract IDVs tab to the table of Keyword search results.

API

Updates and Fixes

  • New Endpoints:
  • Updated Glossary definitions
  • New Sub-tier Agency: National Endowment for the Humanities under National Endowment for the Humanities
  • Advanced Search: Sub-Awards under an IDV are surfaced in Advanced Search

April 22, 2019

Website

Updates and Fixes

  • Various improvements to the IDV Summary pages: the Award Amounts section now intuitively displays edge cases (e.g., obligation exceeding reported ceiling), added minor typographical and edge-case display fixes, and added clear indications of ‘coming soon’ features. Please check back in the coming weeks as we continue to improve the new IDV summary pages!
  • Significantly increased the completeness and accuracy of the IDV type description classifications mappings and related filters on advanced search for IDCs.
  • Improved documentation on api.usaspending.gov

API

Updates and Fixes

April 8, 2019

Website

Updates and Fixes

  • Advanced search has been updated to link to the new contract IDV summary page. The contract IDV summary page will be continuously updated and improved in the coming weeks. We welcome and appreciate any feedback on the new IDV summary page!
  • Fixed monthly award archive generation code to include IDVs.
  • Added Contract IDVs as a separate filter option within Custom Award Download.
  • Fixed minor issue where certain CFDA programs were not surfacing in the advanced search filter picklists.
  • Fixed edge case issue with keyword filter in advanced search related to using a numeric keyword and toggling subawards.

API

Updates and Fixes

  • New endpoint: /api/v2/awards/idvs/funding_rollup/.
  • Updated CFDA loader script and scheduled automatic jobs to keep CFDA reference data up-to-date.
  • Technical improvements
    • Updated Python library versions used by API application.
    • Refactored internally-shared components.
    • Including IDV records in contract monthly archive files.
  • Updated historical transactions for agency subtier code 96CE to point to 096 as its toptier agency.
  • Backfilled Assistance data funding and awarding office names for FY19Q1 and FY19Q2, derived via the Federal Hierarchy.

March 25, 2019

Website

Updates and Fixes

  • Continued backend improvements related to the upcoming IDV summary page.
  • Enhancements to soon-to-be-released IDV award summary page (added federal account funding data and tooltips)
  • Bug fix to custom account download to prevent IDV elements from appearing in the award_type fields and to fix a typo in the agency_identifier header.
  • Added additional glossary terms.
  • Fixed an edge case bug which prevented the small number of unlinked subawards from surfacing in the subaward download, even though they appeared in advanced search.

API

Updates and Fixes

  • Added additional error handling for the Download Service used for all downloads on USAspending.gov
  • Fixing typos in Custom Account Download CSVs
  • Updated Glossary
  • New management commands

March 11, 2019

Website

Updates and Fixes

  • Note: due to the partial government shutdown, FY19Q1 account data will be delayed until 3/21/19.
  • Continued backend improvements related to the upcoming IDV pages.
  • Added IDV subtypes to the Award Type filter on Advanced Search
  • Improved recipient profile name selection logic in the even that names from competing datasets (SAM entry for child, SAM entry for parent, transactional award data for child, transactional award data for parent)
  • Various backend improvements

API

New Features

Improvements

  • Improved logic to determine unique legel entities aka "Recipients" across data sources.
    • Utilizes SAM, and award transactional data to create the entire list of recipient records.
    • Less likelihood of a "child recipient" name replacing the ultimate parent recipient's name.
  • Minor changes to address feedback on API documentation and behavior.

February 25, 2019

Website

Updates and Fixes

  • Note: due to the partial government shutdown, FY19Q1 account data will be delayed until 3/21/19.
  • Continued backend improvements related to the upcoming IDV pages.
  • Updated the recipient profile section
    • Transactions Over Time graph now includes a trendline that provides a count of new awards for each period.
    • Child pages now show multiple parent relations, if appropriate.
  • Expanded the Data Dictionary (available in the Download Center dropdown) to account for the Data Act Information Model Schema v1.3.1
  • Bugfix where Spending Explorer displayed the wrong entrypoint category in certain scenarios.
  • Various backend improvements

API

New Features

Improvements

  • Updated Data Dictionary to DAIMS 1.3.1
  • Improved FSRS data ETL from Broker to USAspending
  • Refactored /api/v2/awards/<requested_award>/ code
  • Moving forward Broker Submissions transfered to USAspending are only copying certified data

February 7, 2019

Website

Updates and Fixes

  • Various backend enhancements and improvements as we prepare to launch updated Award Summary pages, including a new IDV Award summary page.
  • Other minor fixes and improvements (updated external links to Fiscal Service web site, removed small number of subaward data duplicates, remove deprecated React lifecycle methods from the frontend, updated spending explorer to provide clearer sense of when data load is in progress, data dictionary scrolling improvement, spending over time graph year sort order fix, streamlined format of spending by category charts).

API

New Features

Improvements

  • Custom account download supports Navy, Army, Air Force for DOD
  • Sweeping database schema improvements
    • Synchronize production database and Django schemas
    • Standardize datetypes

December 7, 2018

Website

Updates

  • Added the ability to filter by Indefinite Delivery Vehicles (“Contract IDVs”) in Advanced Search
    This allows users to better search contract vehicles that facilitate the delivery of supply and service orders (e.g., Blanket Purchase Agreements).
  • Updated our Homepage to show net outlays for Fiscal Year 2018

API

New Features

  • Now fully support IDV awards in SpendingByAward and SpendingByAwardCount

Improvements

  • Update requests library
  • Corrections to Django ORM model definitions
  • Increased dependability for FABS loader during high-volume loads

October 31, 2018

Website

Updates

  • Updated the Recipient Profile
    Users can now see connections to more than one Parent recipient when the Parent recipient has changed over time.

  • Improved the location filter in Advanced Search
    Users now see results that display only foreign countries when filtering for “All Foreign Countries” as the Recipient location. Users also now see results for all ZIP codes submitted when applying multiple ZIP codes.

  • Updated the Federal Account Profile links on the Spending Explorer
    Users can now go directly to the Federal Account Profile from the Spending Explorer.

API

New Features

Improvements

  • Updated the logic for Custom Account Downloads to bring forward any Treasury Accounts not already included in the most recent quarter’s submission
  • Updated the non_federal_funding_amount field in award downloads to aggregate the total from all transactions
  • Updated file names for Award Data Archive files to include the Fiscal Year, type of download, and creation date of download. This change will be visible to users in November’s archive files.
  • More accurate location (recipient & primary place of performance) scope filters in Advanced Search

October 11, 2018

Website

Updates

  • Added the ability to search by high-level category for account data
    Users can now filter data in our Custom Account Data download based on Budget Function and Sub-Function in addition to filters by Agency and Federal Account.
  • Added a Data Dictionary to our Download Center
    Per user feedback, we now offer a data dictionary that details the data available in our download files, including the definition of each element and corresponding name on the legacy USAspending.gov website.
  • Improved the user experience for downloading data
    Per user feedback, we added a message informing users when the download file is still generating.
  • Updated the links for our Federal Account Profiles
    Users now see a predictable, durable URL based on the federal account symbol.
  • Updated the Higher Education filter for Recipient Type
    Users now see more specific results when searching for higher education recipients under the recipient type filter in advanced search. Specifically, selecting the top-level filter now sums all of the filters below it.

API

Features

  • New API endpoint for Data Dictionary: /api/v2/references/data_dictionary/
  • Include ATA on the TAS string in account downloads
  • New Budget Function and Budget Subfunction endpoints to help filter downloads
    • /api/v2/budget_functions/list_budget_functions/
    • /api/v2/budget_functions/list_budget_subfunctions/

Improvements

  • New CSV filenames for data downloads
  • Replacing boto with boto3
  • Alterations to ETL scripts to be more memory efficient

September 25, 2018

Website

Updates

  • Made sub-agency data easier to find in Advanced Search
    Sub-Agency data now returns as the top result in Advanced Search. Specifically, users can now search and get results for the sub-agency “FEMA.”
  • Updated Award Description fields on Award Summary pages
    Per user feedback, we have updated our logic to display the most relevant award description. Specifically, we now display the description on the first (base) transaction of the award rather than the latest transaction.
  • Added fields additional financial data to our award download files
    Specifically, users can now download financial data related to: Total Dollars Obligated, Inherently Governmental Functions, and Inherently Governmental Functions Description Tag. See our data dictionary for complete details.
  • Improved the user experience for downloading data in Advanced Search
    Users now see the status, “Preparing Download,” while the website assesses if the number of records can be downloaded within Advanced Search.
  • Added NAICS code descriptions for historical codes
    We now show descriptions for historical codes back to 2002 in the Top 5 sections in our Recipient and State profiles and users can view results for these NAICS codes in Advanced Search.
  • Updated the Place of Performance filter for foreign locations
    Users now see more specific results when searching for foreign locations under the place of performance filter in advanced search. Specifically, the filter now checks both the country code and the country name.
  • Added clarifying text to Recipient and State Profiles
    Per user feedback, we added additional text to the Recipient Profile to clarify that the States and Countries Top 5 lists are based on the Primary Place of Performance. We also added text to both the State Profiles and Recipient Profiles to clarify that the bar chart is based on the Transaction Amount Obligated.

API

New API endpoints:

* /api/v2/awards/
* /api/v2/budget_functions/list_budget_functions/
* /api/v2/budget_functions/list_budget_subfunctions/

Reference Data

  • Updated USAspending Glossary
  • Expanded internal NAICS list by including historical records going back to 2002

Updates

  • Updated the download API to have the ability to limit columns for the CSV download file
  • Added ATA to TAS when available for account downloads
  • NAICS description in SpendingByCategory are retrieved from official NAICS records instead of only relying on transaction records
  • Improved Primary Place of Performance location filter for domestic vs foreign countries

September 11, 2018

Website

New

  • Added the ability to search for Federal Account Profiles!
    Users can now search the nearly 2,000 federal accounts by Account Number, Account Name, or Agency.
  • Updated Glossary terms
    The Glossary now includes additional terms related to sub-award data.

Fixes

  • Updated Subaward Data and Subaward Download files
    We added or updated multiple fields, including subawardee_dba_name, subawardee_parent_name, subawardee_parent_duns, prime_award_amount, and highly compensated officer data. We also revised the logic for the recipient name field.
  • Updated the Recipient Type Categories in Advanced Search
    We improved filtering by Business Category at the transaction-level.

API

New

  • Improvements to Subaward data
    • Higher data quality for Subawardee Recipients
    • Additional columns in subaward download files
  • Federal Account search! (new keyword filter in /api/v2/federal_accounts/)
  • Updates to the USAspending glossary source file
  • New README.md

Breaking changes

  • Altered /api/v1/subawards/ response structure. The response JSON doesn't contain as many nested leaf objects, but there are more fields related to the specific subaward in the root object.

August 22, 2018

Website

  • New

    • Added Recipient Profiles!
    • Recipient profiles allow users to explore details about the recipient - including any parent or child recipient relationships - as well as the trend of transactions over time, and multiple “Top 5” summary views. The search page lists recipients in order of the awarded amount in the last 12 months, including lists by award type. These profiles can be accessed from our homepage or the Profiles dropdown.

  • Fixes

    • Updated the Financial Systems Details tab in Award Summary pages The Submission Date column is now populated with the fiscal year and quarter of the submission
    • Updated Subaward Data Subawards were reloaded to fix an issue that caused subawards to not populate

API

  • Added new endpoints for Recipient Profiles
    • /api/v2/recipient/duns/
    • /api/v2/recipient/duns/<recipient-id>
    • /api/v2/recipient/children/<duns>
  • Improved C-D linkage logic when using PIID and/or parent PIID
  • Refactored TAS loader
  • Better recipient data sourced from SAM

August 09, 2018

Website

  • Released a Custom Account Data download
    • Users can download data at the Federal Account or Treasury Account level and filter the data by agency, federal account, and time period. File options include Account Balances, Account Breakdown by Program Activity & Object Class, and Account Breakdown by Award, which links agency financial data to award data. This feature is currently in Beta and we will continue to make improvements to it as we learn more about how users interact with it.
  • Updated recipient-level links in the Spending Explorer
    • Clicking on the recipient name now loads the award results
  • Updated the Keyword explanatory text in Advanced Search
    • This text now shows upon hover in all browsers with an updated list of included fields
  • Added text to Custom Award Data download
    • Additional text now clarifies how to download data prior to 2009
  • Updated the Download modal in Advanced Search
    • The modal now closes once the download is complete
  • Updated the data dictionary link on the About page
    • The link now directs users to the DAIMS v1.2 area of the Data Transparency fiscal website
  • Added a missing icon for Recipients in the Spending Explorer dropdown

API

  • Added repeatable method to resolve Award Financial record linkages to FPDS/FABS Awards
  • Added submission details for Financial Systems Details endpoint on the Awards Profile page to allow compilation of the Submission Date table field
  • Added endpoints to support new Custom Account Download feature
  • Various fixes and improvements

July 18, 2018

Website

  • Added a Database Download option

    • This replaces our Database Snapshot, and is now offered straight from an S3 bucket with a setup guide to help users understand how to download a database of significant size.

  • Updated the text on our Award Data Archive

    • The description now gives more information about how to use our delta files and when to expect updated files.
  • Updated our State Profiles
    • Users can now search by state abbreviations in the landing page. We also fixed an issue that caused the Guam profile not to load.

API

  • Fixed monthly file generation script to include missing parameter of “agency_id”
  • Fixed AWS Region settings variable references
  • Fixed delta file generation to allow simultaneous generation script runs
  • Added SQL based logic for File C to D linkage backfilling
    • This will be used during the nightly ETL process as well

July 06, 2018

Website

  • Added a Recipients view to the Categories tab in Advanced Search
    • Spending by Category now allows users to see the top results by Awarding Agency, Recipient, CFDA Program, and Industry Code.
  • Added a link to our new State Profiles on the Homepage
    • This allows users another way to access these pages and gain insights on the awards performed in that state.
  • Updated the Spending Explorer breadcrumb trail
    • We updated this trail so that it includes Unreported Data when going back up to the top level of the trail
  • Updated the State Profiles landing page
    • States are now listed first with territories included at the bottom
  • Updated the Data Model link in Resources
    • This now takes users to the DATA Act Information Model Schema (DAIMS) area of Data Transparency’s Fiscal Service site

API

  • Performance improvements for the Recipient category for the api/v2/search/spending_by_category endpoint
  • Overall API performance improvements when using location filters
  • Updated api/v2/recipient/state/ endpoint to include type as part of the response object for each state. Possible values: state, territory, district.

June 21, 2018

Website

  • Added Unreported Data to top-level views of Spending Explorer
    • The Spending Explorer now shows the percentage of Unreported Data at the top-level Budget Function, Agency, and Object Class views. Unreported Data can happen when an agency reports incomplete data, an agency has a submission deadline extension, or an agency is not required to submit certain data elements.
  • Released the Categories tab in Advanced Search
    • Spending by Category allows users to see the top results by Awarding Agency, CFDA Program, and Industry Code.
  • Updated download files to include DAIMS v1.2
    • These fields are populated for records that were submitted after 5/18 and fields that can be derived will be populated for historical data later this summer.

API

  • Updated CFDA ETL script based on latests updates from authoritative source
  • Updated api/v2/spending/ endpoint to include unreported data amounts
  • Added api/v2/subawards endpoint
  • Updated download columns to account for DAIMS 1.2 changes
  • Updated api/v2/spending_by_category/ endpoint to change "XX-90" to "XX-MULTIPLE DISTRICTS"
  • General fixes and improvements

June 07, 2018

Website

  • Added State Profiles
    • The profile allows users to view details about the state, an award breakdown by award type, a map of federal spending, the trend of awards over time, and a summary of federal spending through multiple “Top 5” views. The State Profile is based on awards with a Place of Performance in that state.

API

  • Added api/v2/search/spending_by_category/ endpoint to view spending by various category types
  • Added new data load script load_gtas to load expected obligation amounts from SF-133 data
  • Added new endpoints to support the State Profile page:
    • api/v2/state/: Landing page
    • api/v2/state/[FIPS Code]/: Metadata for specified FIPS code
    • api/v2/state/awards/[FIPS Code]/: Award breakdown for the specified FIPDS code
  • Updated subaward ETL loader to handle Award IDs with/without dashes
  • Updated ETL scripts for FPDS and FABS data loads to conform to DAIMS 1.2 updates
  • Various fixes and performance improvements

May 18, 2018

Website

  • Added the ability to submit multiple keywords, recipients, and award IDs in Advanced Search filters
    • Users can now use the filters to search for more than one recipient, keyword, or award ID at a time.
  • Improved performance on the Advanced Search page
    • Users can now consistently see results for sub awards when searching by keyword, among other performance improvements
  • Updated the sub-award loader
    • Users can now see the relevant sub-awards for a prime award regardless of how dashes are used within the award identifier
  • Updated Program Activity display
    • Program Activities with the same name are now combined as one filter on the Federal Account page

API

  • Updating monthly files endpoint to include delta files by default
  • Updated subaward loader to account for dashes in the award identifiers
  • Improved API performance for the advance search page (both prime and subawards)
  • Increased volume of unit and integration tests
  • Updated advanced search API endpoints to support multiple keywords, recipients, and award IDs in search filters.
  • Updated program activity:
    • Updated backend logic to include the program activity name as part of the unique identifiers
    • Updated names in the database to have consistent casing
    • Updated the loader to have consistent casing

May 09, 2018

Website

  • Added Featured Content Section to the Homepage
    • This area will highlight new features and content so that users can stay up-to-date with the latest updates. We are currently featuring the Data Lab, a video tour of the New USAspending.gov, and the option to sign up for updates about the site.

April 25, 2018

Website

  • Added Sub-Award data to Advanced Search
    • Users can now apply the filters within Advanced Search directly to Sub-Awards and see the results in visualizations for Spending by Sub-Award, Spending Over Time and Spending by Geography
  • Added the ability for see the full Award Description within a table
    • Based on user feedback, we added the ability for users to hover over the award description on the Award Detail page to see the full award description.
  • Updated the Recipient Type filters in Advanced Search
    • We updated our recipient type mapping so that filtering by the top-level category of “Business” includes the business types listed below it.
  • Updated the 'total_funding_amount' field in the award download files
    • The field now correctly sums the obligated_amount and non_federal_funding_amount fields.

API

  • Updated the following advanced search endpoints and related documentation to support a flag to filter/retrieve subawards:
    • spending_by_award
    • spending_by_award_count
    • spending_by_geography
    • spending_over_time
  • Performance improvements through updates to matviews
  • Bug fixes to ETL implementation
  • Other various technical improvements and bug fixes

April 11, 2018

Website

New

  • Added sub award download files on the Advanced Search page

Users can now use the filters to customize your search, then click “Download” to get the sub award results. Note that we are currently working on enhancing the display on the Advanced Search page so that users can also view the sub award results (currently the display only shows prime award data).

  • Added the ability to filter by recipient location in the Custom Award Data download

Users can now further customize their downloads to filtering by recipient location. For example, users can now download all sub awards in a specific state.

  • Added the ability for users to sign up to receive the latest updates on USAspending.gov

Users can stay informed with releases and news by subscribing to our listerv on the bottom of the homepage.

Improvements

  • Added clarifying text on Keyword Search page and Federal Account page

We updated the heading on the Keyword Search page to “Spending by Transaction” to clarify that results are at the transaction level, not the award level. We also added the text (i.e., FY 2018 Snapshot) on the Federal Account page to clarify that the Sankey diagram shows data for the current fiscal year.

  • Added additional fields to download files for sub award data

Based on user feedback, we added the following columns to the sub award download files (located on the Custom Award data download and Advanced Search pages): prime_award_principal_place_city; prime_award_principal_place_state; prime_award_principal_place_zip; prime_award_principal_place_country; subawardee_dba_name

Fixes

  • Fixed a bug on the Federal Account Profile

Fixed a bug on the Federal Account Profile page so that quarters in the chart show in the correct order regardless of the filters applied.

  • Fixed a bug on the Spending Explorer

Fixed a bug on the Spending Explorer so that the chart resizes appropriately in Internet Explorer, regardless of the window size.

Operations

  • Improved Download speed

We compressed download files across all download options so that users can download and open smaller files in less time.

  • Fixed a bug related to the Recipient Type filters in Advanced Search

We updated our recipient type mapping so that only businesses categorized as small businesses return when you filter for small businesses.

API

  • Updated the api/v2/download/transactions and api/v2/download/awards endpoints to include subawards
  • Updated the api/v2/bulk_download/awards endpoint to enable filtering by location
  • Updated all download endpoints to utilize enhanced compression to facilitate faster downloads of smaller files
  • Updated the api/v2/download/awards and api/v2/bulk_download/awards endpoints to incorporate additional sub award fields
  • Updated business category compilation logic

March 2018

Website

  • Launched USAspending.gov on March 9, 2018!(Moved from Beta to full production)
  • Added the ability to filter by fiscal year and quarter on the Spending Explorer
  • Added additional information to clarify what data is shown in the Spending Explorer
  • Updated the Spending Over Time chart on the Federal Account page to show multiple fiscal years at a time
  • Updated the Glossary with additional terms and definitions
  • Implemented Accessibility, Privacy Policy, and Freedom of Information Act pages in the footer
  • Added sorting to Bulk Download files
  • Added the Data Lab site link to the header of the website
  • Added Keyword Search into the Advanced Search
  • Standardized headers across the website
  • Fixed bugs related to:
    • Displaying aggregated recipients in the Spending Explorer so that the Recipient view shows each recipient only once
    • Displaying the correct summed amounts for Loan awards in tables and visualizations
    • Updating the logic for Fiscal Year Snapshot Sankey on the Federal Account page
    • Populating the action type on the Award Summary page
    • Updated Award Data Archive files
    • Historical data (pre-2008) for transactions with a blank last modified date

API

  • API endpoint updates to support website enhancements listed above
  • Updated API documentation for v2 endpoints
  • Various bug fixes and performance improvements

February 21, 2018

Website

  • Added the ability to download search results on the Keyword Search page.
  • Released the Federal Account landing page allowing users to view all federal accounts and sort by account number, account name, managing agency, and amount of budgetary resources.
  • Implemented the ability to refresh award data daily (e.g., contracts, grants, and other financial assistance data).
  • Updated styling for title bars on all pages for consistency.
  • Fixed bug on the Contract Profile page related to displaying Indefinite Delivery Vehicle (IDV) contract award details.

API

  • Updated /v2/download/transactions/ endpoint to have a default limit of 500,000 records, which equals the max limit.
  • Various bug fixes and performance improvements

February 07, 2018

Website

  • Added a keyword search feature that allows users to search award data by keyword
  • Redesigned the site home page to highlight the tools on the site, the variety of download options, and reflect 2017 data
  • Implemented a new navigation pane that shows both Award Search Options and a Download Center dropdown
  • Fixed bugs related to:
    • The Loans tab loading on Federal Account Pages
    • Filter summary counts
    • Formatting and linkages of awards in the Spending Explorer
    • Tables within Award Summary Pages and on Federal Account Pages

API

  • Updated Bulk Download endpoint:
    • Limit downloads to 1 year's worth of data
    • Removed retry attempts in SQS when a download job encounters an error

January 24, 2018

General

  • Bug fixes and performance improvements

Website

  • Added hints for the Submit button on Award Search for improved performance
  • Fixed bugs related to:
    • The date range options on Bulk Download
    • Object Class values in the Spending Explorer
    • The display of the Award table when a single record returns
    • Headers on the Agency Landing Page (edited)

API

  • Updated logging infrastructure to include additional logs and support for ingestion through ELK stack
  • Updated list_monthly_files endpoint to remove caching in order to always surface the latest files
  • Updated bulk download endpoints to improve performance

January 10, 2018

General

  • Bug fixes and performance improvements

Website

  • Added the Award Data Archive (https://beta.usaspending.gov/#//bulk_download/award_data_archive) feature to the Bulk Download to enable users to quickly download awards for entire fiscal years by federal agency.
  • Added a Submit button to Award Search so that users select all of their filters before applying them.
  • Improved performance of Bulk Download and added more information to our bulk download messages so that users know what to expect and can copy the link to their download at any time.
  • Fixed bug on the Spending Explorer to enable users to drill down to the recipient and award levels.
  • Fixed bugs related to:
    • IDV contract details populating in the Spending by Award table
    • Searching by location for foreign countries returning awards in Award Search
    • Award details populating on the Award profile pages on initial load

API

  • Added new API endpoint list_monthly_files to support Award Data Archive.
  • General bug fixes and performance improvements.

December 27, 2017

General

  • Bug fixes and performance improvements

Website

  • Released feature on the Award Search page so that users can search by the agency acronym (e.g., NASA) in the Awarding and Funding Agency filters.
  • Fixed bug on the Award Search page so that IDV contract types are included in the search results.
  • Updated the Time Period filter on the Federal Account page so that it only includes relevant fiscal years (i.e., FY 2017 going forward).
  • Made raw agency financial assistance file submissions available (see Raw Agency Files )

API

  • Updated Award Search endpoints to include congressional and county codes with decimals.
  • Updated Spending over Time & Spending by Geography endpoints to include IDV Contracts when filtering on Transactions.
  • Updated Business Type mappings to include additional mappings.
  • Updated Spending by Award Count endpoint to include IDV Contracts and ignore null award categories.
  • Added ability to utilize caching for Bulk Download if identical requests are encountered during the same day.
  • Updated Spending Explorer endpoint to roll up Army, Navy, Army Corps of Engineers, and Air Force into DOD.

December 13, 2017

General

  • Bug fixes and performance improvements

Website

  • Released a tab view of Award Search to make the page easier to navigate.
  • Released new geocoded Spending by Geography maps. Users can now view spending by state, county, and Congressional district for both Place of Performance and Recipient Location.
  • Fixed the subagency search filters on Bulk Download to remove duplicate subagencies.
  • Fixed bug on the Award Search Spending by Geography map so that awards for multiple recipients are mapped to the correct recipient location.
  • Fixed bug on the Award Profile page so that the CFDA program description populates.
  • Fixed bug in the assistance download files to include additional fields that were previously missing.
  • Fixed the Award ID filter to recognize both upper and lower case letters.
  • Fixed various bugs to improve the speed of several Award Search filters.

API

  • Removed dependencies on the Request Catalog.
  • Removed unused database indexes.
  • Removed v1 CSV Download endpoint.
  • Updated v1 Award ID Autocomplete endpoint to be case insensitive.
  • Updated v1 Award endpoint to include CFDA objectives.
  • Updated v2 List Agencies endpoint on the bulk download page to use up-to-date agency list in order to prevent duplicates.
  • Updated Award Search endpoints to utilize inclusive filtering on Award Amount, Recipient filtering to be text based, and added ability to consume Zip Code, Congressional District, and Counties in search filters.
  • Updated Bulk Download to use appropriate CFDA columns.

November 29, 2017

General

  • Bug fixes and performance improvements

Website

  • Released the Bulk Download feature (see link in the website footer). Users can download transaction-level award and sub-award data based on the award type, agency, and date.
  • Loaded and surfaced sub-award data on the Award Profile pages and as an option in the Bulk Download feature.
  • Improved the location search filters on Award Search to include zip code.
  • Updated the "About Page" design and content.
  • Updated Award Profile pages to display additional recipient and place of performance details.
  • Fixed bug so that Recipient Type filters on the Award Search page return the accurate number of records.
  • Fixed bug on the Award Profile pages so that the Awarding Agency and Funding Agency for the award consistently populate.

API

  • Updated award search endpoints to:
    • Consume Recipient Name/DUNS values as a text search instead of a list of IDs.
    • Utilize models for materialized views in order to improve performance.
    • Refactor how IDV contracts are included in search results.
    • Allow filtering by zip code.
  • Updated bulk download endpoint to:
    • Only filter by Awarding Agency.
    • Include IDV contracts when filtering by contracts.
    • Multithread download processes to support multiple download requests.
  • Added various indexes to support performance improvement efforts.
  • Updated subaward endpoint to return the appropriate nested objects in the response.

November 16, 2017

Website

  • Loaded all fourth quarter DATA Act data.
  • Loaded additional award data through 11/16/2017.
  • Released the "Agency Landing Page" and "Agency Profile" page. Users can navigate to a specific Agency Profile page by selecting an agency via the Agency Landing page.
  • Improved the filters on the Award Search page including, Product Service Code, CFDA, Recipient Name, Location, and Keyword Search. Keyword Search matches against the following attributes: Recipient Name, Recipient DUNS, Recipient Parent DUNS, NAICS Code, PSC Code, PIID, and FAIN.
  • Fixed bugs on the Award Profile pages (i.e., Contract Profile, Grant Profile, etc.) so that contract award ceiling is accurate and additional award details are populated.
  • Fixed bug so that the website can be viewed in Internet Explorer 10.

API

  • Refactored Recipient Autocomplete to return a list of IDs that matched to the string or DUNS to roll up duplicates into a single search result.
  • Added database indexes to help with performance.
  • Updated Spending by Geography API endpoint to enhance performance.

November 1, 2017

Website

  • Loaded historical award data (contracts and financial assistance) through Fiscal Year 2010 and surfaced Fiscal Year 2018 data
  • Enabled the “Download” button on the Award Search page so that users can download the results of their query
  • Updated the Spending Explorer to enable users to view results in a table format
  • Made improvements to the user interface for filters on the Award Search page, the Award Summary page, and the Federal Account page

API

  • Added new endpoints to support Award Search Download:
    • /api/v2/download/awards/
    • /api/v2/download/transactions/
    • /api/v2/download/status/
    • /api/v2/download/count/
  • Resolved response caching for v2 endpoints
  • Updated mis-mapped columns for the /api/v2/search/spending_by_award/ endpoint
  • Updated endpoints to enhance performance

October 18, 2017

  • Updated Spending by Award table to filter on Action Date instead of Period of Performance dates
  • Updated Analytics link in the footer to be referred to as Data Lab
  • Updated performance for NAICS autocomplete
  • Resolved filtering based on a specific Awarding Agency
  • Completed loads for Fiscal Year 2017 Transaction data

October 4, 2017

  • Launched Spending Explorer to enable users to explore the new DATA Act financial data. Users can navigate the levels of government spending by choosing their starting point, including budget function, agency, and object class.
  • Updated Search & Download page to be "Award Search" page with the following enhancements:
    • Utilized new v2 API endpoints
    • Added ability to view Spending Over Time by quarters and months
    • Removed Spending by Category option "Budget Category"
    • Added filters for Type of Contract Pricing, Type of Set Aside, and Extent Competed
    • Updated filters to move out CFDA, NAICS, and PSC filters to their own group
  • Updated the Homepage styling
  • Updated footer to add link to Analytics home page
  • Loaded additional procurement and financial assistance data for fiscal year 2017

August 25, 2017

  • Added links to Federal Accounts via Federal Account Name on Financial Systems Detail table on Award Page
  • Added agency abbreviations to all related visualizations
  • Added 404 page for invalid URLs
  • Updated Recipient Business Types on Award Summary pages
  • Updated styling to be consistent for filter pane on Search & Download page across different browsers
  • Updated all links to have consistent styling of blue link text

August 09, 2017

  • Added ability to filter based on CFDA Program, NAICS Code, and PSC Code
  • Updated Award Type filter to include "Other Financial Assistance"
  • Updated Spending by Award table to replace "Insurance" tab with "Other", encompassing both Insurance and Other Financial Assistance awards
  • Enabled every column in the Spending by Award table to be sortable
  • Added tooltips on Federal Account page for Spending over Time and Spending by Category visualizations
  • Added ability to filter based on Parent Recipient/DUNS in Recipient Name/DUNS autocomplete
  • Updated website footer
  • Updated Funding Agency, Awarding Agency, and Recipient autocomplete to provide more accurate results

July 26, 2017

  • Updated Spending by Award table to show results clearly
  • Fixed "See the Breakdown" button on the home page in Firefox
  • Updated site navigation to include additional options
  • Fixed data retrieved based on filters being applied for the Spending by Award table

July 12, 2017

  • Spending by Award Type table displays the number of entries resulting from selected filters
  • Added ability to filter on search page on mobile browsers
  • Selected columns will be included when downloading data from the award table
  • Tooltip added to clarify why sub-functions total more than 100% on the homepage tree map
  • Added ability to remove a group of Object Class filters in the top filter bar

June 28, 2017

  • Minor object classes available as checkboxes underneath major object classes
  • Awarding Agency shown by default on Spending by Award table
  • Updated state filters to allow simple searches (e.g., "TX" for "Texas")
  • Corrected state code information for foreign locations
  • Added detailed information on the place of performance and recipient locations
  • Updated Federal Account autocomplete to allow the notation used in the placeholder
  • Updated Federal Account links to work across all browsers
  • Fixed ability to scroll past the homepage map on mobile browsers

June 14, 2017

Website

  • Allow adding, removing, and reordering columns in the award table
  • Add list view to home page map
  • Add per-capita view to home page map
  • Add category drill down to home page map
  • Fixed tooltip overflow off screen and display on Internet Explorer
  • Add error for results of Funding Agency autocomplete search that contain exclusively Sub-tier Funding Agencies
  • Add total spending metric to tooltip on home page map
  • Utilize new shared checkbox component
  • Fix checkbox filtering expanding automatically to show Sub-tiers when a top-tier box is checked
  • Add Wake and Navassa Islands to US Minor Outlying Islands
  • Update treemap of Budget Functions on home page with drill-down and tooltip capabilities
  • Add meta tags for more information in previews when linking via Facebook, Twitter, and Slack
  • Add support for creating a URL hash based on selected filters and populating filters based on a URL hash
  • Update home page map disclaimer

API

  • Update aggregate endpoints with additional options for handling null values
  • Add /references/filter/ endpoint to retrieve filtering options based on hash input
  • Add /references/hash/ endpoint to create hash based on filtering options as input
  • Removed ability to remove empty federal accounts in the TAS loader
  • Add v2 endpoints
    • /award_spending/recipient to view spending by Recipient
    • /award_spending/award_type to view spending by Award Type
    • /financial_spending/object_class to view spending by Object Class

May 31, 2017

Website

  • Add a data download option to the award table on the search results page
  • Add a search box to the inline glossary
  • Add program activity filter to account page
  • Add Help and Community links to the home page
  • Update tool tip formatting on Spending Over Time Visualization
  • Update category names and formatting on recipient tree map
  • Update funding agency autocomplete to suppress subtier agencies

API

  • Add ability to download and store data in .csv format
  • Add awarding_toptier_agency and funding_toptier_agency to the /tas endpoint
  • Add /recipients endpoint
  • Add /guide/autocomplete endpoint
  • Update field parameter to accept nested fields

Data Loads

  • Performance improvements to the broker submission loader
  • Q2 2017 data has been updated for the Small Business Administration

May 17, 2017

Website

  • Add inline glossary and definitions
  • Add new spending visualizations: by awarding agency, recipient, industry classification, and Catalog of Federal Domestic Assistance
  • Add reason for modification to the contract table
  • Update styles
  • Update About page
  • Update result sort order of Agency search boxes
  • Update warning banner language to be more explicit about what data is available

API

  • Add endpoint to support the glossary: /references/guide
  • Improvements for test suite performance

Data Loads

  • Q2 2017 submission data has been added for Barry Goldwater Scholarship and Excellence In Education Foundation
  • Q2 2017 submission data has been updated for the Department of Justice, Department of Agriculture, and Department of Defense
  • Add data loader for glossary definitions

May 3, 2017

Website

  • Add money in/out sankey diagram to the account page
  • Add object class filter to account page visualizations
  • Add program activity filter to account page
  • Add subawards tab to individual award pages
  • Add executive compensation data to award details
  • Add recipient category treemap to homepage
  • Add tooltips to home page treemaps
  • Update About page content
  • Update assistance award display to include loans and insurance
  • Fixes for visualization formatting on small screens

API

  • Add endpoints for subawards data: subawards/,subawards/autocomplete, and subawards/total
  • Add accounts/awards/total/ endpoint
  • Update recipients/autocomplete/ endpoint to return executive compensation data and better handle missing DUNS information
  • Update tas/categories/ endpoint to return a single set of numbers per fiscal year

Data Loads

  • Update existing transactions when applicable
  • Update broker submission loader to accommodate edge case of agencies using multiple object class records to report de-obligated funds
  • Fix funding toptier agency lookup when funding subtier agency is not provided on the transaction record

April 19, 2017

Website

  • Add budget function filter to the search page
  • Add federal account filter to the search page
  • Add hover to home page treemap
  • Add map to home page
  • Add About page
  • Add quarterly data to account visualizations
  • Update contracts award page to include parent award, not competed reason, and domestic or foreign entity
  • Fix minor visualization formatting issues

API

  • Add endpoint to display awards within federal accounts: accounts/awards/
  • Add endpoints to support quarterly financials by treasury account symbol: tas/balances/quarters/ and tas/balanaces/quarters/total
  • Add endpoints to support quarterly financials by treasury account symbol, object class, and program activity: tas/categories/quarters/ and tas/categories/quaters/total/
  • Add ability to sum by multiple fields to the total/ series of endpoints
  • Add various code descriptions throughout the endpoints
  • Update API doc organization and styling
  • Update accounting data signs for consistent display
  • Update pagination
  • Fix bug that prevented display of transaction obligated amount in the transactions/ endpoint

Data Loads

  • Add 2017 program activity codes
  • Update program activity lookup to include budget year and agency
  • Fix transaction <--> award mapping that was causing display of duplicate award records

April 5 2017

Website

  • Add homepage
  • Add account overview
  • Update search results Spending By Award table to include grant-specific information
  • Update grant details to display Catalog of Federal Domestic Assistance (CFDA) information
  • Update contract details to display Davis Bacon Act
  • Update keyword search to search individual transactions instead of awards
  • Fix miscellaneous minor formatting and styling issues

API

  • Add caching for API requests
  • Add financial_accounts/ endpoint
  • Add endpoints to surface spending by treasury account symbol/program activity/object class: tas/balances/, tas/categories/, and tas/autocomplete
  • Add major_object_class and major_object_class_code to serialized object class information
  • Update tas/balances/ endpoint to return a single set of numbers per fiscal year
  • Update submissions to return reporting fiscal year and quarter
  • Update API documentation to be more beginner-friendly
  • Fix string search

Data Loads

  • Add caching to award load
  • Add federal account load process
  • Fix award update bug

March 22, 2017

Website

  • Add Financial Systems Detail table to award page
  • Update visualizations to display by federal fiscal year instead of calendar year
  • Update search page filters to include award id, award amount, and keyword searches
  • Update contract details to display IDV Type, IDC Type, IDV Agency Identifier, and Multiple or Single Award IDV information
  • Fix various formatting, style, and responsive design issues
  • Fix autocomplete search bugs
  • Fix award table sorting

API

  • Add references/cfda/ endpoint to surface data from the Catalog of Federal Domestic Assistance (CFDA)
  • Add transaction obligation amount to financial accounts by awards data
  • Update awards/ endpoint to include each award's most recent transaction
  • Update API docs to include a complete endpoint list, recipes, introduction for new users
  • Fix case sensitivity in string searches
  • Fix full text search regression
  • Fix performance issue caused by page metadata/pagination

Data Loads

  • Add updated version of agency list, including agency abbreviations
  • Add various performance fixes

March 3, 2017

Website

  • Add awarding and funding agency filters to search page
  • Add recipient filter to search page
  • Add visualizations to search page
  • Add hovers to Spending by Geography map on search page
  • Add award contract page
  • Add analytics tracking
  • Add contract amount visualization
  • Fix label and formatting issues

API

  • Add a single transactions/ endpoint that returns information common to all award types (contracts, grants, etc.)
  • Add an API landing page and style the API documentation
  • Add API request logging
  • Update awards/ response to remove nested account activity. There is now a separate endpoint for retrieving that data: financial_activities/award/
  • Update totals/ series of endpoints to handle aggregations by fiscal year

Data Loads

  • Add awarding agency to award mapping criteria
  • Update program activity and object class loads
  • Fix issue with updating a legal entity (i.e., award recipient) when it already exists
  • Fix date parsing error
  • Fix a loading issue caused by invalid states

Website Documentation

Search Page Visualizations

Federal Account Page

Clone this wiki locally