2020
Search results
Predsednik Narodne stranke Vladimir Gajić i novinar Zoran Ostojić bili su gosti "Info večeri" televizije Informer. Govorili su o brojnim temama od velikog značaja za Srbiju. 24.06.2024
- Planeta
Advokat Goran Petronijević dobio je, tokom gostovanja u...
- Sport
Kako nezvanično saznaje Informer, neki od najvažnijih...
- Live TV
Gost "Info večeri" televizije Informer bio je akademik...
- Hronika
Hronika "Mahnula si mi i rekla: Doći ću ja tata ponovo"!...
- Mundobasket
Košarka Novo lice u Partizanu: Od rata i izbegličkog kampa...
- Foto/Video
Redakcija Informer.rs zadržava pravo izbora, brisanja...
- Đilasovac predviđa krah opozicije na Vračaru i Starom gradu: Oni tu neće formirati vlast (VIDEO)
- Šok! Šta su ovo uradili Kinezi i Rusi?! Amerika gori nakon neočekivanog manevra velikih sila, ugroženo glavno oružje za vladanje svetom!
- Saša Ćurčić bez dlake na jeziku za Informer: Belingem je mafijaš, Englezi se jedino plaše Mitra! Kad vidim naš spisak
- Danas do 33 stepena! RHMZ se oglasio pred vikend: Spremite se na još veće vrućine i tropske noći!
- Konačno je neko ukapirao! Ćuta na tajkunskoj N1 priznao: Problem je opozicija, a ne Aleksandar Vučić! (VIDEO)
- Planeta
- Overview
- ProbSparse Attention
- Requirements
- Data
- Reproducibility
- Usage
- Results
- FAQ
- Citation
- Contact
This is the origin Pytorch implementation of Informer in the following paper: Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. Special thanks to Jieqi Peng@cookieminions for building this repo.
🚩News(Mar 27, 2023): We will release Informer V2 soon.
🚩News(Feb 28, 2023): The Informer's extension paper is online on AIJ.
🚩News(Mar 25, 2021): We update all experiment results with hyperparameter settings.
🚩News(Feb 22, 2021): We provide Colab Examples for friendly usage.
🚩News(Feb 8, 2021): Our Informer paper has been awarded AAAI'21 Best Paper [Official][Beihang][Rutgers]! We will continue this line of research and update on this repo. Please star this repo and cite our paper if you find our work is helpful for you.
The self-attention scores form a long-tail distribution, where the "active" queries lie in the "head" scores and "lazy" queries lie in the "tail" area. We designed the ProbSparse Attention to select the "active" queries rather than the "lazy" queries. The ProbSparse Attention with Top-u queries forms a sparse Transformer by the probability distribution. Why not use Top-u keys? The self-attention layer's output is the re-represent of input. It is formulated as a weighted combination of values w.r.t. the score of dot-product pairs. The top queries with full keys encourage a complete re-represent of leading components in the input, and it is equivalent to selecting the "head" scores among all the dot-product pairs. If we choose Top-u keys, the full keys just preserve the trivial sum of values within the "long tail" scores but wreck the leading components' re-represent.
Figure 2. The illustration of ProbSparse Attention.
•Python 3.6
•matplotlib == 3.1.1
•numpy == 1.19.4
•pandas == 0.25.1
•scikit_learn == 0.21.3
•torch == 1.8.0
The ETT dataset used in the paper can be downloaded in the repo ETDataset. The required data files should be put into data/ETT/ folder. A demo slice of the ETT data is illustrated in the following figure. Note that the input of each dataset is zero-mean normalized in this implementation.
Figure 3. An example of the ETT data.
The ECL data and Weather data can be downloaded here.
•Google Drive
To easily reproduce the results you can follow the next steps:
1.Initialize the docker image using: make init.
2.Download the datasets using: make dataset.
3.Run each script in scripts/ using make run_module module="bash ETTh1.sh" for each script.
Colab Examples: We provide google colabs to help reproduce and customize our repo, which includes experiments(train and test), prediction, visualization and custom data.
Commands for training and testing the model with ProbSparse self-attention on Dataset ETTh1, ETTh2 and ETTm1 respectively:
More parameter information please refer to main_informer.py.
We provide a more detailed and complete command description for training and testing the model:
We have updated the experiment results of all methods due to the change in data scaling. We are lucky that Informer gets performance improvement. Thank you @lk1983823 for reminding the data scaling in issue 41.
Besides, the experiment parameters of each data set are formated in the .sh files in the directory ./scripts/. You can refer to these parameters for experiments, and you can also adjust the parameters to obtain better mse and mae results or draw better prediction figures.
Figure 4. Univariate forecasting results.
Figure 5. Multivariate forecasting results.
If you run into a problem like RuntimeError: The size of tensor a (98) must match the size of tensor b (96) at non-singleton dimension 1, you can check torch version or modify code about Conv1d of TokenEmbedding in models/embed.py as the way of circular padding mode in Conv1d changed in different torch versions.
If you find this repository useful in your research, please consider citing the following papers:
If you have any questions, feel free to contact Haoyi Zhou through Email (zhouhaoyi1991@gmail.com) or Github issues. Pull requests are highly welcomed!
Informer su NAJTIRAŽNIJE dnevne nezavisne novine. Budite uvek u toku sa najnovijim i ekskluzivnim vestima iz politike, društva, ekonomije, džet seta i rijalitija.
The meaning of INFORMER is one that imparts knowledge or news. How to use informer in a sentence.
Dec 14, 2020 · To address these issues, we design an efficient transformer-based model for LSTF, named Informer, with three distinctive characteristics: (i) a ProbSparse self-attention mechanism, which achieves O(L log L) in time complexity and memory usage, and has comparable performance on sequences' dependency alignment. (ii) the self-attention distilling ...
- Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, Wancai Zhang
- arXiv:2012.07436 [cs.LG]
- 2020
Informer is a British television drama series, created and written by Rory Haines and Sohrab Noshirvani and produced by Neal Street Productions for the BBC. The six-part series stars Paddy Considine, Bel Powley, Nabhaan Rizwan, and Jessica Raine. The series began broadcasting on BBC One on 16 October 2018.
2 days ago · Game Informer is your source for the latest in video game news, reviews, previews, podcasts, and features.
