[david.tauriello@xbrl.us]

Hi Hasnain – if I understand correctly, you’re trying to display the human-readable label AND the corresponding fact value.

If you know the dts.id and concept.id, you can pass these as /label/<em>dts.id</em>/search?concept.id= to get the label.text value.

Take a look at the XBRL Statements and Disclosures template for Excel (be sure to unhide columns A-E and rows 1-4).

IMPORTANT: the task pane in Excel is a very cut-down version of filter options available – keep our XBRL API documentation handy!

[david.tauriello@xbrl.us]

Hi Christian – thanks for writing; we posted Simple Statement and Disclosure template for Excel that uses /report /fact /relationship and /label queries to pull this information together.

Take a look at the queries buried in hidden columns and rows A1:E4 of that .xlsx. In a nutshell:

• /report (optional) – filing details like entity name, type of report and date filed (cell F7); you DO need a dts.id integer and can get this from this object
• /fact – numerical value, year and period information for concepts in a specific part of the report
• /relationship – filtered on the presentation linkbase* of the taxonomy, where the ‘face’ of the financials is stored
• /label – the human readable value for taxonomy concepts that are associated with facts

*remove this from the query to show facts from all linkbases (calculation, definition, label and presentation)

IMPORTANT: an XBRL filing is made up of an instance document that contains all the facts, and a taxonomy that establishes the structures (linkbases) for those facts.

[Zoraiz Azeem]

As I have recently been given access to Request to use OAuth2 for XBRL API via email.
My account is linked to my google account, but when I try to set a password, via link : “https://xbrl.us/set-password”, I continuously get an error, asking to try again after a few minutes. I have tried multiple days in a row and still no luck.

How can I set a password for account, without this I will not be able to do a POST request as demonstrated in the documentation provided.
Thank You in advance.

[david.tauriello@xbrl.us]

Zoraiz – sorry for your troubles; I’m not sure what the issue is and will follow up with our team next week.

For now, try to register for a web account using the same email that was provisioned for you to use with OAuth2 and the XBRL API.

Please post back here to let us know if this works for you.

[Zoraiz Azeem]

Hi David thank you for your quick reply,

I have retried the “https://xbrl.us/set-password” link again and it was successful but only when I tick the generate password automatically box. I then do receive an email in my spam. And after changing the password, and retrying to log in with said new password I for some reason receive a “wrong password or username” message. I have tried to use the “new” password with the post request but recieve a ‘{“error”:”invalid_request”,”error_description”:”Bad username or password.”}’
message.
I may just try and provision a different email that is not a gmail.

Thank you.

[Peter W Reed]

Tim I have the same experience as you. I have found label mapping in the “XBRL TAXONOMY EXTENSION LABEL LINKBASE” file to be the best source for mapping common names with XBRL names. Below is an example from Microsoft’s latest 10-Q filing. It maps “us-gaap:NetIncomeLoss” to the common name “Net income”. The Python package “xmlschema” is the best library I’ve found to parse Edgar xml files. BeautifulSoup does a great job too. I prefer BeautifulSoup due to familiarity and the time it takes xmlSchema to execute (15 seconds) is too long.

To get the history you mention parsing needs to be done on each individual filing, yuck. Even then additional data cleansing is required. I know a better way is out there.

<link:label xml:lang=”en-US” xlink:label=”us-gaap_NetIncomeLoss_lbl” xlink:type=”resource” xlink:role=”http://www.xbrl.org/2003/role/totalLabel”>Net income</link:label>

[david.tauriello@xbrl.us]

Hi Zoraiz – please try to register for a new account using your current email address.

[Mikko Olkkonen]

Tim,
I am developing my own special purpose standardization for few tags that are relevant for my specific use case. I uploaded my mapping into my github repo. I am interested in similar mappings that I could add into my version.

[Tim Bui]

Thank you for sharing the info on label mapping, Peter! I am reading this topic now. I also need to learn BeautifulSoup so that I can parse the xml files.

[Peter W Reed]

I can upload snips from my program to github, if you think there is value. The code is sloppy and meant for personal use only. But it would be a good start with BeautifulSoup. The xmlschema library is very easy to use. I will include code snips for xml parsing too. My assumption is you are familiar with Python data structures like Pandas and default dictionary.

Below is a sample output of a default dictionary using a list factory. This is from Walmart’s Q1 2016 10-Q filing. There are a total of 216 keys. Below is a small sample of us-gaap mapping to common names (list):

‘stockholdersequityincludingportionattributabletononcontrollinginterestabstract’: [‘Equity:’,
“Stockholders’ Equity, Including Portion Attributable to Noncontrolling Interest [Abstract]”],
‘commonstockvalue’: [‘Common stock’,
‘Common Stock, Value, Issued’],
‘additionalpaidincapital’: [‘Capital in excess of par value’,
‘Additional Paid in Capital’],
‘retainedearningsaccumulateddeficit’: [‘Retained earnings’,
‘Retained Earnings (Accumulated Deficit)’],
‘stockholdersequity’: [“Total Walmart shareholders’ equity”,
“Stockholders’ Equity Attributable to Parent”],
‘minorityinterest’: [‘Nonredeemable noncontrolling interest’,
“Stockholders’ Equity Attributable to Noncontrolling Interest”],
‘liabilitiesandstockholdersequity’: [‘Total liabilities and equity’,
‘Liabilities and Equity’],
‘comprehensiveincomenotetextblock’: [‘Accumulated Other Comprehensive Income (Loss)’,
‘Comprehensive Income (Loss) Note [Text Block]’],
‘incomestatementabstract’: [‘Income Statement [Abstract]’],
‘revenuesabstract’: [‘Revenues:’, ‘Revenues [Abstract]’],
‘salesrevenuenet’: [‘Net sales’, ‘Revenue, Net’],

[Tim Bui]

Please upload your program to Github, Peter. I would learn a lot from it. I started to learn Python and Pandas and can get around Python comfortably. Would you also please send out your Github link? Thank you!

[Peter W Reed]

I placed the code snippets in a readme for a new project. Creating a .py file didn’t make sense since it wasn’t a full script or algorithm. Please find a link to the readme file below.

XBRL_label matching.

[Tim Bui]

Thank you, Peter! I am studying the codes now

[Peter W Reed]

The xmlschema code snip will fail as written. It worked well for me because I had already defined the header in the BeautifulSoup program. I’ll edit the github to include the header definition. The SEC has an acceptable use policy that forbids (error code 403) the use of automated web scraping programs. The default header for request used in BeautifulSoup and urllib used by xmlschema identifies itself as a Python header. I believe they did not intent to prohibit programs using Python under user control like Arelle, or the programs I posted on github.

Use the header definition provided below. Note there are Python modules that perform more sophisticated header rotation. Edgar doesn’t seem to need that level of spoofing to work.

headers = {
“Host”: “www.sec.gov”,
“User-Agent”: “Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Firefox/89.0”,
“Accept”: “text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8”,
“Accept-Language”: “en-US,en;q=0.5”,
“Accept-Encoding”: “gzip, deflate, br”,
“DNT”: “1”,
“Connection”: “keep-alive”,
“Upgrade-Insecure-Requests”: “1”,
“Sec-GPC”: “1”,
“Cache-Control”: “max-age=0”
}

• This reply was modified 2 years, 8 months ago by Peter W Reed. Reason: typographical errors

[Peter W Reed]

Be careful with the xmlschema output. The xbrl parsing is correct, but printing the output is precarious. Multiple styles are used in creating the 10-Q/K. The output “xmlTmp_xbrl” is a dictionary but the keys and other data structure elements change year to year and company to company. The number of variations is small but a test for which variation is in use needs to be determined.

[Author]

Posts