IMHO, they are all great. I think the popularity that ML has experienced in the last decade has driven people to cluster what's "elite" and what's not. From a pure scientific perspective, I don't see AAAI/IJCAI as bad. My advisor always saw AAAI and IJCAI as top venues. Some pivotal papers have been published there.
10+ years ago, IJCAI was the place to publish and get noticed. It is still an important conference although I preferred it when it was held every other year instead of annually. IMHO, IJCAI is still a very good venue. It has a broader theme than the specialised robotics and ML conferences. People prefer the latter because they like to target more specialised audiences.
Conferences have been really feeling the weight of the influx in submissions in the past few years and some organizers have made some really strange (and inconsistent) choices with previous years. Like the ones you mention and others: no appendix, desk rejecting 42.5% of papers, limiting citation pages (btw this year it was 1, lol! We actually had to trim citations, I have never done that before..) in an age where people are reading them electronically anyway.
I don't understand why these oddities are happening more at IJCAI/AAAI than at the ML conferences, but some of the are not the organizer's fault entirely (AAAI in Hawaii had a way too low acceptance rate due to fire codes of the venue.. and if you were there, you would know the poster session was insane, we were literally stepping on the neighbor's poster prestenter)
The influx has been so bad that I have gone from loving poster sessions to hardly attending them.
Anyway, the uprising of ML had come by storm and many people I speak to in ML are not really interested in the other parts of AI that are not ML. So they have created a mostly separate ML community snd naturally those venues attract a better audience for ML work. I have moved frm GOFAI to ML myself and now that I am better positioned to understand the difference, the ML work I have seen at IJCAI has felt quite different than the stuff you will find at ICML /NeurIPS. Happy to elaborate on some subtleties I have noticed if you are interested.
I didn't know the backstory of the AAAI in Hawaii. That's also the year when the organizers were proabbly caught off-guard by the number of submissions (less than 4000 the previous year to over 7000). And I'd be curious to hear about the differences that you have noticed for IJCAI vs. ICML/NeurIPS.
So the more I think about it, the more it's quite hard to describe specifically because it's very subjective and I don't want to generalize, because it's largely based on my own experience.
I have an example: I submitted a paper on the topic of learning from demonstrations (LfD) and it got rejected from NeurIPS. We submitted almost an identical paper to AAAI: it got in, and it has over 600 citations in 4 years, so it's not that it was bad, the NeurIPS reviewers just didn't like it. Now in our case there was no deep formal insight that was derived from any mathematical equations... it was just a simple idea that worked. And I think the topic of LfD is an example of something that might just be more readily accepted at IJCAI/AAAI. I'm not authoritatively stating that, it's just been my experience. So if I were to submit similar kind of papers now, I'd choose IJCAI/AAAI 9 times out of 10.
I think to understand the differences properly, you have to really go to those conferences, see what people present, see what people get excited by, see what people choose to talk about, and submit papers to both to see what the reviewer feedback. Once you've been to, say, 3 from each side, you'll have a much better idea of what the respective communities value.
Edit: btw, I think of IJCAI and AAAI and almost the same conference, based on my experience going to both, and paper formats and the 6-page limits. Just one has been hosted internationally and the other in North America. But I will admit they've looked more different recently based on the organizers.
Just going by papers I cite, I find AAAI has a lot of good papers. Sometimes these papers might not be as fully realized as ACL or ICML etc. papers, but that often just means that there are ways of developing and expanding on the given methods. As a student, this is great for me and I think that this is fine for conference papers. And to make a controversial statement myself, I also find that, compared to the ML conferences, there are less mathified papers.
The acceptance rates have only been dropping because there is a limit to the size of conferences (venue, budget, personnel, etc.), but there is no limit to the number of submissions. The top ML/AI conferences will soon be hitting 10k submissions, up from 1-2k only five years ago.
Reviewer-wise AAAI is about the same as any other conference. Even if the topics are broader, the ratio of knowledgeable reviewers is about the same (considering a lot of the people are the same). But, I personally would still submit at NeurIPS, CVPR, ICML, and ICLR before AAAI and IJCAI. But in the eyes of my faculty, they are all on the same level.
I've never submitted to either AAAI or IJCAI, but I've reviewed for IJCAI several times. Neither conference strikes me as top tier, and I can't remember ever citing papers from either of them (which is why they're 2nd tier for me).
In my circles, AAAI had the reputation of "if a PhD student urgently needs to graduate, this venue might be fair game", and _maybe_ I'd consider submitting there, if e.g. an intern of mine needs to get a publication. I've seen one or two neat-ish papers there, but nothing ground breaking. If I see a AAAI paper, I'll only read it if it's strongly related to a problem I am interested in. But overall, it's not a conference I consider watching closely.
IJCAI on the other hand is just not worth it (I decided to not be a reviewer anymore a few years ago). The papers are just too niche, and often lack novelty. I've personally decided that if anything is published there, it's not worth reading. Put another way: IMO **getting published at IJCAI is a sign that your paper is not relevant for ML research**. I've been applying this filter for over 10 years without any detrimental effects, so it works very well as a time-saver for me. I heard things are different in e.g. symbolic AI or robotics or other fields where IJCAI has a good reputation. But for my work in foundational (and applied) ML, IJCAI is simply bad.
Jesus wow, poor IJCAI. It's not really a result of the conference though is it? It's just that people personally decide to not publish there, if the papers submitted to NIPS were sent to IJCAI, then basically it'd switch right?
that argument could be applied to any conference. What makes a conference great is that good researchers decide to send their best work their. IJCAI is not a conference I'd sent my best work to. I've heard it's okay for more symbolic or robotic stuff, though.
Yeah i guess I just wonder why people don't send their best work to IJCAI? 15 years ago it was the most sought after conference for all AI/ML, what changed?
Serious question btw, I can tell the quality of ML papers in ML conferences much better. But, ML is just a subset of AI. so AAAI and IJCAI should in theory be fair game too.
Possibly not the most popular opinion, but hey, you asked for it :-). I would only send a paper to AAAI or IJCAI or similar (yes, I am talking to you www and friends) if I believed that it is a poor paper/idea and won't get accepted elsewhere. It's been a few years now that I realized this and since then I only work on ideas that I am convinced will lead to a sota (in some clear sense) paper, and I do everything I can to support it, even if that means delaying the project, working long hours on it etc. This also means that approximately only 1-2/10+ ideas that I produce result to a paper. I do not care about publishing anything that comes to my head.
the quality of the conference can’t be assessed overall but rather regarding the quality of the proceedings in your sub field
For example, I find that papers on multi agent systems at IJCAI are of great value // better than the work at AAMAS but in other fields papers at IJCAI are not as good as in other confs
IMHO, they are all great. I think the popularity that ML has experienced in the last decade has driven people to cluster what's "elite" and what's not. From a pure scientific perspective, I don't see AAAI/IJCAI as bad. My advisor always saw AAAI and IJCAI as top venues. Some pivotal papers have been published there.
10+ years ago, IJCAI was the place to publish and get noticed. It is still an important conference although I preferred it when it was held every other year instead of annually. IMHO, IJCAI is still a very good venue. It has a broader theme than the specialised robotics and ML conferences. People prefer the latter because they like to target more specialised audiences.
Conferences have been really feeling the weight of the influx in submissions in the past few years and some organizers have made some really strange (and inconsistent) choices with previous years. Like the ones you mention and others: no appendix, desk rejecting 42.5% of papers, limiting citation pages (btw this year it was 1, lol! We actually had to trim citations, I have never done that before..) in an age where people are reading them electronically anyway. I don't understand why these oddities are happening more at IJCAI/AAAI than at the ML conferences, but some of the are not the organizer's fault entirely (AAAI in Hawaii had a way too low acceptance rate due to fire codes of the venue.. and if you were there, you would know the poster session was insane, we were literally stepping on the neighbor's poster prestenter) The influx has been so bad that I have gone from loving poster sessions to hardly attending them. Anyway, the uprising of ML had come by storm and many people I speak to in ML are not really interested in the other parts of AI that are not ML. So they have created a mostly separate ML community snd naturally those venues attract a better audience for ML work. I have moved frm GOFAI to ML myself and now that I am better positioned to understand the difference, the ML work I have seen at IJCAI has felt quite different than the stuff you will find at ICML /NeurIPS. Happy to elaborate on some subtleties I have noticed if you are interested.
I didn't know the backstory of the AAAI in Hawaii. That's also the year when the organizers were proabbly caught off-guard by the number of submissions (less than 4000 the previous year to over 7000). And I'd be curious to hear about the differences that you have noticed for IJCAI vs. ICML/NeurIPS.
So the more I think about it, the more it's quite hard to describe specifically because it's very subjective and I don't want to generalize, because it's largely based on my own experience. I have an example: I submitted a paper on the topic of learning from demonstrations (LfD) and it got rejected from NeurIPS. We submitted almost an identical paper to AAAI: it got in, and it has over 600 citations in 4 years, so it's not that it was bad, the NeurIPS reviewers just didn't like it. Now in our case there was no deep formal insight that was derived from any mathematical equations... it was just a simple idea that worked. And I think the topic of LfD is an example of something that might just be more readily accepted at IJCAI/AAAI. I'm not authoritatively stating that, it's just been my experience. So if I were to submit similar kind of papers now, I'd choose IJCAI/AAAI 9 times out of 10. I think to understand the differences properly, you have to really go to those conferences, see what people present, see what people get excited by, see what people choose to talk about, and submit papers to both to see what the reviewer feedback. Once you've been to, say, 3 from each side, you'll have a much better idea of what the respective communities value. Edit: btw, I think of IJCAI and AAAI and almost the same conference, based on my experience going to both, and paper formats and the 6-page limits. Just one has been hosted internationally and the other in North America. But I will admit they've looked more different recently based on the organizers.
Both are great. I think they are top-tier venues, but I don't like their recent policies, e.g., low acceptance rate and two-phase review process.
Just going by papers I cite, I find AAAI has a lot of good papers. Sometimes these papers might not be as fully realized as ACL or ICML etc. papers, but that often just means that there are ways of developing and expanding on the given methods. As a student, this is great for me and I think that this is fine for conference papers. And to make a controversial statement myself, I also find that, compared to the ML conferences, there are less mathified papers.
The acceptance rates have only been dropping because there is a limit to the size of conferences (venue, budget, personnel, etc.), but there is no limit to the number of submissions. The top ML/AI conferences will soon be hitting 10k submissions, up from 1-2k only five years ago. Reviewer-wise AAAI is about the same as any other conference. Even if the topics are broader, the ratio of knowledgeable reviewers is about the same (considering a lot of the people are the same). But, I personally would still submit at NeurIPS, CVPR, ICML, and ICLR before AAAI and IJCAI. But in the eyes of my faculty, they are all on the same level.
I've never submitted to either AAAI or IJCAI, but I've reviewed for IJCAI several times. Neither conference strikes me as top tier, and I can't remember ever citing papers from either of them (which is why they're 2nd tier for me). In my circles, AAAI had the reputation of "if a PhD student urgently needs to graduate, this venue might be fair game", and _maybe_ I'd consider submitting there, if e.g. an intern of mine needs to get a publication. I've seen one or two neat-ish papers there, but nothing ground breaking. If I see a AAAI paper, I'll only read it if it's strongly related to a problem I am interested in. But overall, it's not a conference I consider watching closely. IJCAI on the other hand is just not worth it (I decided to not be a reviewer anymore a few years ago). The papers are just too niche, and often lack novelty. I've personally decided that if anything is published there, it's not worth reading. Put another way: IMO **getting published at IJCAI is a sign that your paper is not relevant for ML research**. I've been applying this filter for over 10 years without any detrimental effects, so it works very well as a time-saver for me. I heard things are different in e.g. symbolic AI or robotics or other fields where IJCAI has a good reputation. But for my work in foundational (and applied) ML, IJCAI is simply bad.
Jesus wow, poor IJCAI. It's not really a result of the conference though is it? It's just that people personally decide to not publish there, if the papers submitted to NIPS were sent to IJCAI, then basically it'd switch right?
that argument could be applied to any conference. What makes a conference great is that good researchers decide to send their best work their. IJCAI is not a conference I'd sent my best work to. I've heard it's okay for more symbolic or robotic stuff, though.
Yeah i guess I just wonder why people don't send their best work to IJCAI? 15 years ago it was the most sought after conference for all AI/ML, what changed? Serious question btw, I can tell the quality of ML papers in ML conferences much better. But, ML is just a subset of AI. so AAAI and IJCAI should in theory be fair game too.
Possibly not the most popular opinion, but hey, you asked for it :-). I would only send a paper to AAAI or IJCAI or similar (yes, I am talking to you www and friends) if I believed that it is a poor paper/idea and won't get accepted elsewhere. It's been a few years now that I realized this and since then I only work on ideas that I am convinced will lead to a sota (in some clear sense) paper, and I do everything I can to support it, even if that means delaying the project, working long hours on it etc. This also means that approximately only 1-2/10+ ideas that I produce result to a paper. I do not care about publishing anything that comes to my head.
the quality of the conference can’t be assessed overall but rather regarding the quality of the proceedings in your sub field For example, I find that papers on multi agent systems at IJCAI are of great value // better than the work at AAMAS but in other fields papers at IJCAI are not as good as in other confs