Saurik: What to And What Not to Expect From the Next Version of Cydia

By , Mar 15, 2011

In a recent post about the next version of Cydia, we got some great feedback from users about what features they’d like to see in Cydia 1.1.

The creator of Cydia, Jay “saurik” Freeman posted a long and informative response to our article that gave some insight into the current and future state of Cydia. He chose many questions posed in the article’s comments to address key issues.

We’re very grateful that saurik decided to share this information with us. He addresses several key topics concerning Cydia…

Features of Cydia 1.1 will include:

  • the ability to run and operate Activator, libstatusbar, and SimulatedKeyEvents while Cydia is running
  • an overall speed improvement, including the “Loading Changes” dialog
  • “much less” memory usage
  • a more advanced search mechanism with a new relevancy algorithm
  • better management of broken repositories

Here’s everything saurik had to tell us about Cydia:

Multitasking

“This is simply not possible at this time. I realize everyone wants it, and hell: I want it, too; but everyone saying it is important doesn’t make it possible. The reason Cydia doesn’t have iOS 4 compliant multi-tasking is that, to make the modifications it makes to the system, it runs as “root”, a user that has more permissions on the system than anything else, which means that SpringBoard, a lowly process running as “mobile”, cannot suspend/resume it.

Now, this is something that /can/ be remedied, and is something that I’ve been thinking of how to do for a long time, but all the obvious ways people like to bring up for making Cydia’s GUI run as mobile with only small parts running as root would make Cydia run slower, and speed is something that is primary on everyone’s minds when they are using Cydia. Luckily, there are things I’ve figured out that may make this more reasonably possible, but certainly not for this release.”

Leaving Mobile Substrate On When Cydia is Open

“If you did this you would find your system would suddenly become unusable. I realize that it sounds all nice and fuzzy that Mobile Substrate should modify all applications on the system, including Cydia, but again: Cydia runs as root. Almost all extensions in the ecosystem are NOT designed with this in mind, and when suddenly given root access start destroying the permissions of your configuration files and Media folders, making all normal applications unable to use them.

Therefore, with this release of Cydia, I’ve gone through the “big ticket items”–which seem to come down to Activator (what starts SBSettings), libstatusbar (adds notification items to the status bar), and SimulatedKeyEvents (injection of key events from Veency)–to verify with their developers that they will work correctly in an environment running as root. These extensions (plus WinterBoard, which doesn’t work on root on 4.x but is harmless, and will be fixed in a future release) are what are available from inside of Cydia until Cydia is modified to run as mobile.”

Better Looking Interface and Backup Option

“A backup function actually does not require pushing a new build of Cydia, but it requires time to figure out how to scale the users to support the kind of load Cydia has. Cydia is running with many orders of magnitudes more users than any of its competitors have, which means that a lot of things that people like to think “should be simple, X did it” are actually much harder to implement. I also keep privacy at the forefront of my mind while building features like this, and want to be 100% certain that no one can get access to your installed products list other than you.

As for a “better looking interface”, I try hard to maintain something that competes with Apple’s products. A few things rotted on 4.x (the positions and sizes of some buttons), and the various “black” interfaces (the black bar and the black screen) get mixed reports, but otherwise the main problem users have with Cydia is not Cydia: it is with repositories. Every time I’ve gotten actual feedback “this specific thing is bad”, where that thing isn’t something that Apple themselves do in their iTunes or App Store applications (which should be taken as the “intuitive model”), it is in areas of the interface I simply do not have any control over: the content shown for a package by the repository.”

Confusion With the Term “Changes”

“Maybe I’m crazy, but I always thought of the word “Changes” to be a very non-geeky end-user term for “stuff that changed”. It certainly isn’t a technical term: it was not chosen because of some geeky desire to have the codebase match the UI, nor was it chosen because it had some esoteric meaning in Latin or Greek. It was instead chosen as it was a single word that immediately meant to the largest number of people I talked to exactly what that page did: showed you what changed. Regardless, “New Releases/Updates” certainly won’t fit on a tab label.”

Speed

“As with /every/ release of Cydia, Cydia 1.1 is faster than previous releases. In specific, it is faster than 1.0.3366 by a good margin, which itself was faster than 1.0.3222 by an even larger chasm. On this note, however, it is important to note that Cydia is tackling a hard problem: no other application I have seen on the iPhone, from Apple or any third party, is attempting to search index and manage tens of thousands of data items, on the client, in real time, aggregated from user-selected sources.

In contrast, Cydia has some of the fastest technology in existence with regards to handling this data, whether it be custom algorithms (Cydia includes a locale-aware string comparison radix sort, which AFAIK is the fastest sorting algorithm in any iOS application) or special on-disk data structures (new in 1.1 is “Cytore”, a new way to store local metadata on packages that can be loaded nearly instantaneously from flash; for those out there who are technically minded, it is an on-disk memory mapped hashtable, which drastically beats out alternatives people like to try to bring up such as SQLite).”

Loading Times

“Despite myths to the contrary, the amount of data displayed in the Changes list does not drastically affect how quickly it loads. There /was/ a bug in many versions of Cydia 1.0 that caused there to be at least a little delay related to the number of items on the list, but this bug was already fixed as of 1.0.3366. The cost of the calculation is deciding what entries should be on the list at all (and specifically which ones are actually updates vs. new releases), not displaying them all at once. That said, Cydia 1.0.3366 moves the loading of changes until after you click the tab, which makes it more evidence how much time is being spent on this feature (which itself is, again, faster on 1.1).”

Memory Usage

“Despite Cydia 1.1 continuing to attempt to juggle tens of thousands of items in memory, thanks to Cytore, it uses much less memory than ever before. Other optimizations have been made, as with every version of Cydia, in order to decrease the memory usage of the app as a whole. Additionally, and in particular, Cydia 1.1 is much more conscientious of memory warnings, and attempts to throw out as much state as possible during these events.

That said, the amount of memory on even reasonably modern devices (anything past the iPhone 3G) available for running applications (not in total, but available after Apple’s system applications get their share), is an order of magnitude greater: whereas on an iPhone 3G you were working with maybe 20MB of available memory, on an iPhone 3G[S] you have 150MB, and an on iPhone 4 you have 400MB available. So, despite Cydia 1.1 actually needing less memory to operate than Cydia 1.0, the pressure on memory is pretty much gone, and will not affect future users thanks to hardware upgrades.”

Advanced Search

“Unfortunately, this device is simply too slow to provide “advanced search capabilities”, and certainly not suggestions, given the constraints of “from user-selected repositories” “in something resembling real time”. That said, Cydia 1.1 has a much better search mechanism, including an integer-arithmetic radix-sorted relevancy algorithm I managed to implement.

What would really be needed to have a truly amazing search experience is to not do searching on the client: to instead handle it on my servers. This is how products like the App Store, Kindle, or Netflix work: it is not at all common for services users are used to to attempt to manage the entire database /on the device/, doing local searching, rather than having the data and computation for that existing in offline-indexed search structures on a massive server in the cloud.

Unfortunately, the reason people use Cydia are varied, and many people are using Cydia with repositories that frankly they shouldn’t be: whether the repository contains software that is dangerous (a niche community with tweaks receiving minimal testing, or using bad practices like on-disk file patching) or downright illegal (there are things you are allowed to do in your country that I cannot in mine), I am certainly not going to be acting as the centralized storage and indexing gateway for people to find and manage this content.

Instead, what keeps people coming back to Cydia is the fact that it acts as the fundamental alternative: that rather going to Apple, with their carefully curated set of centralized experiences, you go to Cydia, “the wild west of software”, where software modifies other software in a kind of reckless abandon that is going to lead to pain even in the best possible scenarios, and in the worst possible worlds is going to lead to things that you will not be able to list on a default repository, and which Cydia may even warn you about installing, but which you should still be able to access and even search for using Cydia’s search mechanisms.”

Error Messages

“Errors from Cydia do not come from Cydia. If you type a URL into Cydia for a broken repository, that repository is going to be low-quality and is going to cause you problems. If it is offline, Cydia is going to tell you it is offline, and if it is malformed Cydia is going to get angry about that. Cydia is simply going to sit there idly while there are a ton of broken and offline repositories in your list: it will tell you all of the errors involved in the hope that you will remove the broken repositories and get on with your life (which is a very apt metaphor, as most third party repositories are very slow, and cause your refresh experience to take a very very very long time).”

“Rate” and “Review” Sections

“We actually tried this, and it was a miserable failure: more time had to be spent moderating the reviews, most of which were misleading, inflammatory, or downright inappropriate, than anyone got value out of this mechanism: it was even worse than on the App Store, which is notorious for bad reviews (people often rate down a package for inane reasons, making the data horribly invalid).

Given these issues, I attempted to put together a vision of how comments and ratings could work in Cydia, and even made a trial implementation (screenshots were even handed out at some points, and I did demos at a few conferences), but when word came up that I was even considering releasing it, I received strong pushback from some of the best developers in the ecosystem–the people you are most likely to want to give mega-good reviews to–that if I continued with that they would give up on the ecosystem, due to the issues from before.

And, to be honest, I am not certain that I would have solved those problems, and given subsequent experiences from alternative products, and looking at how people used the ratings, what people said in the comments, and how things finally got rated, I no longer believe that I would have: I believe the concept of the off-the-shelf “comments and ratings” to be a fundamentally flawed system that inherently leads to abuse.

Now, not all rating systems need to be “off-the-shelf”, so something truly innovating and “actually solving the problem” is what I hope to one day provide for Cydia. In the meantime, however, I always do my best to avoid injecting seriously suboptimal tradeoffs into our ecosystem.”

Improved Compatibility Listings

“Cydia has, for a while, contained numerous features that would allow repositories to help with this problem.

  1. a mechanism to specify firmware compatibility on packages (packages can Depends: specific firmware revisions).
  2. the Cydia Store lets vendors block purchases for specific firmwares (any paid product can register its compatibility with its repository, and then I will filter it to users who can use it).
  3. the firmware version is sent as part of the user-agent to the web pages for each product, allowing developers to display their own warnings.
  4. compatibility is even more specially able to be done by feature detection, allowing packages to say “I need voiceover support on a device with a camera running an armv7 CPU and a retina screen”.

In essence, there is very little excuse for packages, repositories, products, or anything else in the Cydia ecosystem to be poorly specified in terms of firmware compatibility. That said, almost no packages in the ecosystem, and even very few products (where one would imagine this to be the most important), have this information included at any of these levels, which is rather disappointing.

So, Cydia 1.1 is not going to attempt to improve on any of these mechanisms, as Cydia 1.0 already has more than enough of them: the real onus is now on the developers and artists of specific items.”

Removal of Dead Content

“I do not have any control over what content is available in Cydia. I mean, I can refuse to personally accept money for it, but I have almost no introspection over things that are either free or sold on the developer’s website. For years I have attempted to get repositories to pull obsolete packages: they refuse. Instead of lobbying me, who agrees with you and is powerless, you need to be sending these complaints to the default repositories: BigBoss, ModMyi, and ZodTTD.”

Clarified Install Requirements

(Tags saying if installation of apps/tweaks, etc. need a springboard refresh or if the device needs to reboot)

“While this is often stated, this is simply not how this mechanism works: packages compute whether they need a reboot or reload as they install, allowing packages to make optimizations like “I only need to reboot if the user is using this firmware version and has this other package installed with this setting specified”. In fact, all of my packages that need features like this attempt these optimizations, and often you will not need as many reboots or reloads because of it.

Therefore, specifying this as static tags on a package would increase the number of reboots a user has to perform needlessly. That said, for packages where it is not obvious (extensions are going to require a reload, and MobileSubstrate is going to require a reboot), such as cases of MMS clients that require a reboot, it should certainly be best practice for the developer to put this information on their package information screen. This is even easier for the developer/vendor than modifying the package, and even then is very uncommonly specified: adding the tag therefore won’t even change how often it is reported.”

Repo Management

“With regards to repositories that did not add correctly, as stated by Ryan Petrich, Cydia 1.1 should no longer end up in situations where broken repositories are so unusable that they are also undeletable. That said, many users complain about repositories installed via a package: to delete these repositories you will need to remove the package that represents them.

(Due to some of these complexities, it is Cydia policy going forward that no repositories will be installable from default repositories via packages, and the existing ones under More Sources will be transitioned to a new mechanism for handling these that has been added that will allow more direct, simpler, and safer manipulation of repositories using a soon-to-be-revamped More Sources page.)”

Saurik’s closing comment,

“Thank you all, by the way, for your interest in Cydia: the fact that you care at all about what features are or are not in Cydia 1.1 means a lot to everyone working on the project.”

We want to sincerely thank saurik for his detailed and informative responses to our reader’s questions. We wish he and his team the very best in making Cydia the best it can be!

Hopefully you learned a few things from this lengthy discussion. Feel free to discuss or share opinions about what saurik had to say in the comments below.

  • Share:
  • Follow:
  • Burge

    to go along with the longest reply to a post..I think that this is the longest post..
    From what I read Cydia is working the best way on our devices and that saurik is trying to improve on that .. Keep up on the good work …

  • lewisbey

    Saurik, your the man! everything you do with Cydia is appreciated. Doesn’t always work perfectly, but it makes my IOS life much more enjoyable with Cydia. I love tweaks!

    Jailbroken devices in my household: 3 iphone 4′s, 2 ipads, 3 Atv2′s. And lovin it!

  • IndiePhoenix

    Thank you, Saurik, for your great work! Cydia is already great as it is. Thank you for make it even greater! :)

  • Xepptizz

    A very respectable reply and well argumented. Makes me more appriciative of cydia. That’s for sure.

  • Tr0jan69

    Changes and Updates have the same amount of letters! I’m ok with it being changes though :p

  • QuarterSwede

    I’m impressed. He adressed every one of my grievances with excellent explanations. Kudos Mr. Freeman. It’s quite obvious you’re doing all you van for this community. Hats off to you.

  • jailbreakFTW

    Saurik: iPhone users around the world thank you for helping make the JesusPhone everything it should be.

  • Leeloo24

    Simply thank u. Having read all the comments from the previous post I really felt like he addressed them ALL! Can’t wait for the release :)

  • feelthepain444

    God was i happy when cydia took over installer… that said, ill be even more happy with that new update :D

    There’s one thing still unclear for me though about the reboot/respring requirements… Why can’t you make the kind of requirement like Icy did? ie. after installing any package which would normally require reboot/respring, you go back to cydia and can still install other stuff.

    sometimes it can get annoying respringing everytime you install stuff (i know there is the queuing option but still..)

    Anyways, keep up the good work!

    - A truly dedicated iPhone Hacker

    • TheShortcut

      I agree with you fully on that.

  • Fredy

    Saurik: Loading… Downloading Packages… Reloading Data… Phew!! .. I just wanted to thank you so “thank you”! I look forward to seeing a Taylor Swift like Cydia in the future :)

  • Rainrose

    Thank you Saurik for Cydia. :)

  • Sig3rs

    Its a best tweak store/centre in the world..thank a lot..

  • babeh

    nice comments from the one who has more 4 phones online at the same time.
    Keep the good work, saurik.
    Worried about aslr though

  • Vitica

    Whatever it is……thanks a lot Saurik!
    You’ve made my iPhone more enjoyable and Fun…
    Also thanks to Alex n IDB
    Keep up the good work!!

  • greatblog

    Just thought I should let people know that with the app Backgrounder you can set up Cydia to run in the background and multitask.

    • Faester

      I was just about to write the same!
      Backgrounder is wonderful.

      Thank you saurik for Cydia! Where would we, the not-very-technical-jailbreakers be without it? It only inspirers me to become more advanced in the jailbreak world – because Cydia makes it all easy. With these words I think we’re ready for a tv-commercial for Cydia.. Lol, just keep up the good work!

  • Thor

    Thank you for all your hard work and great explanation of issues.

  • Saman

    Thank you Saurik for Cydia

  • Michael

    When will this be released? With the untethered 4.3 jailbreak maybe?

  • brian089

    how do i get 1.1 cydia

  • Damen

    Awesome! Great work, much faster and snappier. Is there a way to turn of badges for Cydia? That would be a great feature. thx again.

  • penis

    who can I download cydia ??

  • Ray

    Thank you to you and your team.